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CHAPTER 1 



Overview 



Comparisons ofrprogratrTjs, institutions^ and state systems pf 
higher plication are often made. Sometimes these corr^- 
pansons strongly influence imporjant^-decisions, sometimes they 
amount to little more than rhetorical gestures, But the trend is 
toward the former— and toward more cojnpg^risoijis. Therefore, 
.comparative analysis 'i^ a proper concern/>f administrators and 
pdicynjakers throughput American higher education. But few 
adequately understand its appropriate role and weighrth decision- 
making- Hence this book, w^hich b concerned with the preparation 
and use of data needed to make objectivej usefial comparisons, 

Gcner^ principles as well as specific issues ^r e d be ussed, in* light of 
.two major premises that arise from the authors' experience with 
comparati^ a natysis \m higher pd ucation. T+ie first is tKat tITe use of 
comparatLveM^ta by institutions, system administratidns^and state 
agencifes frequently is reasonable and proper, perhapjs unavoidable. 
Our secopid^remtse is that serious problems can attend the.Use of 
comparative data and that evea tn^he best of circu;nstancest no 
^.omparison is perfect. These assertions rnay strtke the reader as 
transparent truisms^ safely made about almost any +iurrfan,activfty. 
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CHAPTER 1 



\Vc suggest tfiac thc\ hj: regarded somewhat like doors, ta^ily ' 
st*en throtigh, but noy^wuhour subetancc. ^ 

At ?hL^ outset, Lumpafativedaca must he distinguished from tom- 
pjfatue informal k)h. Dtj£*i-^a\ be quantities, numbers ttiat 
represent measurements. Or data. may f>c eneL>deJ observations, 
signify ing bv a ^Vmbul the preM;nec or absence of eertam eharactcr- 
i^tKi. ^Enrollment dataware re<;ordcd as numbers; a student*s 
religious preference is recorded'b> a code J lnformatior\ consists of 
data chat are usefcl in one or more contexts, that inform "someone 
about ^mcthing^ that reduce uncertainty in some way. To become - 
useful informjition/the data may havo^^to be combined in one or 
more ways or displayed in a,partKM*kT form> though this is not 
al^^aviii the case. To determirfe vvhetRer or not a planned level of 
overall institutional full-time equivalent (PTE) enrollment has been 
rt'ached, only one number, a simple totah i^- needed. To know 
something abn^ut the pSittern of increases srnd decreases in enroll- - 
m^int in the venous Student-ma)or prog-rams offered in colleges of 
arts and sciences, the total enrollment figure must be brokentiown 
to departftient-le\d numbers. Then each of those numbers must be 
compared to its counterpart number fbt the previous year. To know j 
iomethmg about the trend in part-time student enrollment, total 
PTE enrollment musj be compared tw head-count enrollrnent, those 
numbers can be broken dou n (disaggregated an analyst would say) 
and the segments manipulated in various ways to produce many- 
dim^^nsioned information. ' , 

The important ramifications of the distmction between data and 
information are cogently discqss^d at more length in another 
volume in , the l^CHEMS Executive Overview series-7-Datd and 
InfoTmation for Hxecutit^e Decii(ons in Hi^Vr Education^ by Dennis 
Jones (1982)-,,Her^, we f<^us on data in their role as the basic * 
building blocks for most formal— jhat i^, objectively groanded and 
analytically disciplined— comparisons in higher education. We also 
have a good deal to say about tlj'c procedures, processes, and 
political elemerrts that can foster jmpede the tikelih'bod of 
cohnparati^e. Jata actually becoming useful information for 
adm?nistrarors, \ ' * ' 
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The Case for Comparative Datjf 



Cumparanvcdatj arc being used c\temjvcl\ *it this time in hight,T 
edui^jtum for sc\cral rcai>ons. First, an J most ohMOUsly, more data 
are available nov\ than ever before. The gra^u'al adoption of the 
view that data can bei^ome information useful for most planning 
and management actuines, and the now wholesale adoption of 
Lumputeri:ed record-keeping systems, have created rhe abundance. 
The increased reliance on data for administrative purposes in 
colleges and univ^^ities has-been motivated by both internal and 
cxteirial factors, ^'ithm institutions, recognition of the value of data 
at>.iut institutional performance has grown steadily. In the 4950s 
and 1060s^ the increased awareness came largely because of rapid in- 
i^reases in the size of institutions. In the 1970s> financial stringency 
bcL^me a strong motivator. Institutional grovtth and then themulti- 
pl^atjon of finani^Ial problems and complexities brought about ^n 
incred^e in the number ofpfiid-le^vel administrators— institutional 
researc^her^, planners^ assistant deans, assistants to the' presidents, 
financial^aid specialists, and so on. In most cases, their prirtijary 
activities involve the produc;tion and wse of data. 

The extemah deTnarrd for dajta: has 7Tiun^T:bm-h:prpaix7to-t 
point svhere toda\ tlie typical i^ollege or university admini^trat<on 
feels overburdened with reporting tasks. From government and 
from pn\atersector Sources alike, the requests ketp^oming: data are 
a^^q^jired about affirmaCiva action, student aid, apd general statistics 
on the federal side, state-tevel agencies want data relatiog to finan- 
cial allocation and program distribution; the lobbying activities of 

-higher-education 'associations call for another set of nurnbers; and 
the data ne^Js ofthose who study higher education are steadily 
expanding. (At present, there are more than 80 graduate |3rograms 
[n the field of higher education, and any number of sociologists ^J^d 
economists are busy anal^ijiing the ^^S^havior of colleges and 
universities.)^ ' |' 

/ Nonetheless, demand has not outstripped supply, A great plenty 
of data remains a^hand, much of it Adaptable, atTeast in principle^ 
to comparative s^nalysis. Commonly made comparisons incl^ide 
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t.n.C»1t\ salaries at unc jn^ititution Ms-a-x i^ those at p<?cr institutions, 
stat^''^b\ 'State appropriations p^;r student, and the relati\ c quality of 
graduate programs- * 

Antithcr reason k^v the sCcadily cxpandinit; jj^c of data in higher 
^.-du^^atkm Is the M^nif^ant ad\ ant.es made in the standardization of 
data ^.olleiwtcd b> and from institutions. Standard definitions of lii^a 
Y dementi, tfor c^JampIt;, fidlumi: studem) and widelv a'w^:eEfied ' 

'a^LOunting proi^ed^res are ne(:essities if (:ompar4iO*^5'^^re to be 
substantive and reliable- Nearly three decades ago» Russell and Doi ^ 
0^55), two pioneers in institutional analysis^ spoke of a ^'trying need 
tor reliable normative data on the expenditures of institutions of 
higher edui^ation" (p- 21). That need has yet to be fially met, either 
tor finan«.ial data or for Other bods. But ^:onsiderabIe progress has 
been mad^.^— nv.>tably the agreement reached in the mid'I970sby the 
National Association of College and University Business Officers 
^ iNACUBO), the Ameri^:&n Itistitute gf Certified Public: A^:(wOun- 
tanti lAlCPA), and th^; National Center for Higher Edu^:ation 
Management Systems (NCHEMS) that Aa^pdardised revenue and 
,^^e*ipertditure categories and financial ae^wOunting procedures general- 
ly. So,tuday*s greater amounts of data have been rrtade more 
Tinlilorn^ xvKKTTmearis that'sc^nd comparative analysis is mO^-e, 
^ ' attainable than ever before, . ^ * 
' Finally, the p<;rCei\ed need for corjiparative data h^s grown apace 
• with the increase in viability of the data. To be sure, this need is 
f p^^r^:elved differently from various \'antage points in the highly 
diverse higher-eduiwatton community- Howevetj some broad trends 
arQ-enhant.ing to the utifity of comparative data, "The most impor- 
tant are increased systemiiatton and coordination of higher educa- 
tiun, mure emphasis on performance measures and accountability, 
greater interest irt long-range and strategic planning, and^the 
, incrt^ismg a%^areness of a need for greater management control* 

Th(LMi«Cfmou5 expansion of Ameritan'highef education in this ^ , 
' v.enturyj esfJecially since World War 11^ has been accompanied by a 
steadily increasing nepd 'to rationalize the allocation of resources 
and the division oF responsibilities ^||fcng institutions. This has 
been most markedly ttue in the public sector^' of coursej but many 
private institutions have f^'t the influence of chrs trend. The mt}st 
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\ ij^iWo pi^Ktwar ni<rnifc;>t.ition of chc tr^^nd ha.s been the proliferation 
n^itionallv of stiue-levcl <.oordih;ftmg boards and consolidated 
l^i^AtTnin^ bu*ird^. Oi^e or tlie other «.an be found in every state 
e^vepr Delaware, Wmiont', and Vi'voming. While publK institu- 
tiun^ are the mam Li.>nLern of ^.wrdi nation-efforts, the interests ^nd 
^.ontnbutJons of pn\ate institutions to a scates higher -t^dueat ion 
re^oUTLC often are given signifiLant cunsidtratjon when pi31icy is 
made- The authority of Loordinacing agencies varies widely from 
state tustate^^bui^the agenLie^ seem to have in Lummun a large appe- 
tite for Lomparative data on the mstitutiuns \MJthin thejr respective 
purMcwi, HUrold Enarsun tl979), former president of ^hio State 
Unuersit\, has noted that the desire on the part of state govern- 
ments tu aLhie\e L-qujty in funding is a major reason for the exten- 
M\e use of comparative data in the context of state coordination. 
' In publiL and private institution^ alike, administrators always 
have been viewed a^ stewards of the resources they manage. But a 
pujSlic accounting^ at regular intervals has not been a part of the 
American tradition jn higher education; iniact, some prestigious 
private institutians have long treatc\J the president's annual report 
to t^ie trustees as confidential. But open accountability has become 
the generat ruIe*over the past two decades, at least in the publi^ 
sector, and private institutions in financial distress have been more 
than willing to document the extent of their trouble. The key rdlof 
higher education in the socioeconomic life of the natujn means that 
jt must be gi\en substantial support. During the protracted period of 
financial stresi and strain affecting all levels of governments, the 
relative performance of colleges and universities inevitably will 
temain under close scrutiny. This scrutiny extends to-.questioris of 
effectiveness— that is, \i,hether institutions are doing what is right 
and doing it well— as vvell as to questions of efficiency , which probe 
into how economically institutions are able to function, * * 

Demand for accountability arises both internally and externally, ' 
Faculty have an abiding interest iri their compensation and working 
conditions, in the quality ^ incoming students, in the adequacy of 
library resources, facilities^ ^nd equipment, an^l so on, Stgdents are 
interested tn the quality of instructional progtams, degree require* 
ment^; thp institutions placement record, and such. The regents or 
^ » ^ ^ 
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f>roL:r.im^ jiid^th^.^ pL'rform*HKc of die pr<."iidcnt, Externatlv- all * 
tunJorv Ud\i. ^Ik' rj^)it tu ^tf^ aLt^kUintin^ for vv!iat is done with ^lio 
ri. vi>uri.LS iht:\ prov ld^.^ F<.^u^ aJniinKtr;Kars arc+^ostilc to the pnn- 
oi a^.^.oufltdl>llltv^ [Jut manv an- vvarv of providing K through 
lucdKim oi omparayy^' datj, lx^au>c thc\ know chat the mo>t* 
ucll-meaUing ^.^.)TT^arL^on t.an be dangerous(^y mislcadingt 
^ NV^ncthdos, the demand for^omparaci\cda(aoftj^ is t.ompe[lc<^ 
h\ thtf ahsun^e of abiulutc J>tandards against whith (o asses^ ii^trtu- 
tional [^rfo^man^.e^ It is not inherently ob\ious what the minimum 
level of student outt^omesor the student-^faeulty ratio ought to be, or 

' h^)w bro.i*d the vurneulum ought tobe, or how much the ifistitution^ 
shi^uld pd\ faculty or charge students, or whetljer a particular 
dep.irtment *,>r mstHution|s rfece?!^'ing its "fair share** of resources, A 
4,onsen.\Ui. may Be operating with respec^ to' extremes; a it'u/lcnt- 
fa^^ulty i\uitj/rt LQOJ surely wouUbe uniyersally regarded as unac- 
^^(.^ptahlc. But gb^jA^r^^ is no consensus about appropriate 
. standar^Js^or Ie\\^]swmnT^ the range ofoptions open tcvmost mstitu- 
tioiii^ moj^t of tne time. A few rule-of-thumb stan'dards for certain 
resv,iu/i.es or .capacities— square feet df classroom space per student 
^.redn hout, for e\gmple— are widely ac^^eptt^, ^hese usually wete 
ptumulgated h) dp (accrediting body or professional society. In most 
instan^^es, the origin ors^ich rulesorstandards can be traced to com- 
parative data— usually data on the behavior df institutions ihoi^ght 
to be exemplary, ' " , ' * 

III the aby^nce of absolute fram^es of refer<^nce, higher education 
hab no choice but to rely on relatite standards to some extent. The 
Mtwation is rn some respects ana logous-to' that in the cbsihcte, -where 
tcTnpv,tral and spacial location can be determined and expressed only 
in r|^arional terms. The idea of a relativity principle for higher ' 
edui.ation may ^ound cjffaint, but it does seem to\cflect^a basi<i 
feature of reality in the worfd of'colleges and Universities, The 
behavior of other instrfutior^, their structure, performance^ flind 
methods of operation, both -present and p^t^ may constitute an 
imperfect and somew hat protean framework for comparisons. But it 
IS nune'theless a framework withm whic*fi usefial understanding can • 

^Be gained aboutor^e s own institution. This certainly iS not t|)e only 
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*kmd of , institutional understanding that is, needed or can be 
iihtajntd, noj HeLCSsarily the mt^st important kind. But for certain 
i^^uo, Lumparatuc data ,.an he the ^uur^^e of valuable lYisights not 
ottjer^^nse obtairtable. 

Three more points should be made about (comparative dq^a and 
acLOuntability! Firsts the demand for a(:countability in the absen(:e 
uf standard performariLc measures (:auses a perennial need for com- 
paratKc data. Some of the evaluative ^atJb*fcqutred to, establish 
aLLeptable performance measure? will necessarily inv6lve'(:ompari' 
son&, ScLond, the need for v^onriparativedata in this context is strong 
enougb to\irtually guarantee that sooner or lacer, comparisons will 
be made among departments, itistitutions^ states, and even nations^ 
uhether or 'not those being compared d^ire it or carry out the 
analyses themselves. . ^ ^ 

The third point is a matter of authorial judgRnent* We think that 
on tbe whole^ itkbetter for the institution (or'othef unit of analysis) 
to take the initiative in the comparison process^ or at least tQbecome 
a \vi!ling partiLtpant. Taking tnfo account the complexities and 
potential ^problems tha^urround the use of conjparatiye ^at^i it 
iieldom *^ ui&e to [et^omeofkc-else dictate che terms— the 'framework, 
a&siimptions, dimensions— of the comparison. Not to (:ooperate 
uithland possibly LO-opt) the process is to risk being unfd^irly treated 
.md ipbjeLteii to misleadin^comparisons. This eventuality usually 
leadi to a reactive mode and absorption with damage controh 

So far, ue have portrayed (:omparative data as mainly a conae;' 
queni^e of developments that'impin^e upon adn>inistrators whether 
they Iik^ ttor not* But it is really not so one-sided. Aftfer all^ the very 
finaciLiat data requested by a coordinating agency may provide an 
institution justification for asking for addit^al fUnds. Moreover, 
comparative data can assist an administrator iij holdm^ others 
accountable, establishing perfortnanct^ standards, «nd accomplish- 
ipg other management-control tasks* Comparative data can* be 
especially helpful to administrators engaged tn strategic planning* 

By Urattgn. planning, \^e^mean essentially the planning and assef^* 

ment that relate to institutional mission: What role in the large 

scheme of thin^ can the institution (program, department) expect 

to play? What niche can it c5ccupy in the marketplace? What services 
K 
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'ihoulJ rendered, and for'iji'hon^^ Vt'hat are the jnstitiJtion*s^ 
str^-^nj^K^and woaknosses^ Nearly alldepartments and institutions 
L-ngagf m thm' kinds of ^.onsjderjtions to varying degrees. However, 
some institutions under puhliu uontrpl are preempted frorn doing 
^^xtensue strategiL*pIannjn^ be^:ause sjtate legislators, coordinating- 
board ^^maly sts, or system-level administrators m effect do it for"^ 
them. Some jsrestigious institutions, on the other hand, are so ell 
established inlthe marketplace, and so set in their ways, perhaps, 
that disLUssiorls of new or modified strategy would be superfluous, 
^ But fup instltufiohsseekin^an identity, or some change fn ttjeir role 
and scope to ensure survival, strategic planning has strong appeal* 
(George Keller [1983] prv)v'i4es some excellent commentary on 
strategic planning^ higher edWarioo,) 

In any case, 'comparative d^astan support or be integrated into 
strategy planning insevetal ways. Gathering and analyzing data on 
other providers and*on the enterprise as a, whole are part of the 
t^mnonrner^l ^nmng^spect of strategic planning. Such scanning 
yields data particularly relevant to goal setting. The comparative 
analysis reveals a range of empirically possible performance, pro- 
viding a framework for en^visioning an institution's fijture in con- 
*wrete terms, Sifnilarly, the targeting and poiitioning aspects of strategic 
planning often can be facilitated by^ appropriate comparative data, 
ladeed, the very nature of comparative advantag^^ which every 
strategic assessment should try to identify, requires that it bje^ 
assessed through comparative analysis, * ■ * 

Trfhigher education, some of the more common compansons are 
akm to this type of strategic assessment. Tuition setting, when it is 
an institutional prerogative, usualFy will int^olvenporcthanan inter- 
nal assessmentof revenue requirements. Tuition rates at other insti; 
tutions nearly atw^s are given at least some consideration. And 
'some evidence suggests that comparative data^on tuition rates' 
influence institutional tuition policy in the public secoor (Rusk and 
L^eslft 1978). The same holds for'the prices an institution is wiJHng to 
pay for the Services of faculty and staff: the institution's position on 
salaries and wages relative to comparable private-sector pay levels 
t will mske a difference m the caliber ofpersorfr>el that the institution 
, can hire and retain. : 
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The c\aluatjve Side of Strategic pl^nnm^g can alsp make usf; of com- 
^parati\e data. The usual question js this: Are we doing well or 
puort> in this or chat area of endeavor? Internal data getierally pro- 
vide onl^ part of the answer.To'take a simple example, suppose that, 
an institution has experienced growth in enrollment for some years. 
On Its fa^e/this trend would sean to reflect well on the instttutjon.* 
But much more will be revealed if the growth pattern is analyzed 
within the context of enrollment behavior at similar institutions. 
makes a jJifiference^ both for understanding the past and for foretell- 
ing t|^e fiature^ whether an institution has been riding a wave or has 
somehow been able to move against.a current- Again^.the basis for 
insight h relative performance: ^ ^ 

Thtre afe good reasons, then, for using comparative data 'ici 
^ higher-eduuatjon administration and management, even if it is not 
^required. The rational allocation of resources and responsibilities , 
the estSibltshment of a viable framework for accoun^tabtlity and 
management coptrot* and the implemeritation of several aspects of 
^ strategic planning depend in part on comparative data. 



Caveats * . * 

^ ^ ' r-r^ ^ 5-^ 

c ha^!? said more than once that using c<3mparatlve data has its 
perils. Indeed* there are compelling reasons for not doing some 
kinjs of ^comparisons, at least among institutions or among states* 
discifes data-related problems in detail later on* Here we will 
make only general comments about the perils of narrowness, 
misrepresentation, sameness, mediocrity, mtsperception* and 
' costliness, ..^ . * - ' *• 

It IS one thing to useVomparative data as a source of supplemen- 
tary int^ormation, quite another to use it as the c6re of consideratitm. 
In the latter case, comparative data can promote a naf rowness of 
understanding and outlook. This might not be so if comparative 
data \^ere availa^^ble regart^ing all conceivably important aspects of 
all colleges and u'niversities. lo-Jgct] th^y are not' available to 
^nywhere near the ideal extent. So in proportion as reliance On 
comparative (^ta increases, the field of vision tnay decrease. If 10 
— ^ . 
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dimcriMon*. of (wompari^on are important, but data are available on 
only 4, then the importance of the other 6 comparisons v/lU 
sv,)m^,'Bmeii be forgotten. Conscientious analysis and use of com- 
parati\ c data always proceed w ith lively avi areness that the data are 
never as i^oiviplete as'|they,might be— and sometimes are only bits 
and pii^^ej.of what oitie would like to have. * 

On o^i^asion*' narri|)wness falls into outright misrepresentation* 
Higher education js fjarticularty vulnerable to some types of corf^- 
parisons because of iti long-staR^ing difficulty in assessing the out- 
corfies of the educational ptocess- (Howard Bowen [I980J usefiilty 
summarizes the predicament.) To the extent that the outcomes 
remamunknown or unmeasured, assessments o(both efficiency and 
^Ci^tivenesi are challengeable. Properl^ constructed, such assess- 

^ mi;nts would ^^onstitute a type of cbSt -benefit analysis. Knowing just 
the ^ost3 involved is noG enough. Nor is it enough to asse^ output 
solely in terms of quantities produced— the numbef of degrees 
■granted or studertt credit hours generated. Instructional cpst per 
student Credit hour may bean interesting r^io to some, but its utility 
in comparative analysis is oft^n open to challenge, Harold Hodgkin- 
ion,^a leading light irXresearch On higher education, once observed 
that ''no one has ever felt,smelled,or heard^ or seen a credit hour. A 
i^redit hour is simply a measure of time spent in a place of instruction^ 

, in the presence of some instructional medium" {1976, p. 41)* Unless 
we know^ the value of a credit hour {which varies, of course, with 
perspet^tive), we cannot reasonably infer from th^ cost figure very 
mu^h about the efficiency or effectiveness of the educational pro- 
cess. Again, as With the 100:1 student-fac^lty ratio we hypothesize^!, 
cost figures that are many multiples of those at peer institutions area 
dear caust* for alarm* But wTchin-the normal ran^ of cost experi- 
ence, the value of such comparisons is questionable at best and the 
chame of misrepresentation js high. Misrepresentation is pafrticu- 
larly^ threat when a number ofpartJes and interests are free to inter- 
pret the data, including some having little faroiliarity with th^ aims 
and operations of the higher-education enterprise (iJicNeil 1972). 

^'hen institutions are gauging their beha^ior^gainst that of other 
institutipns,-an unwelcome drift toward sameness may resuU. Diver- 
sity m fcigher c4y'-ation traditjonally has befti vali^ed highly. If the 

10 
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u^e'of Loonparative data promoccjb micrnLry, more harm than good 
mci> rVsulc. But such data ^re used properly la a strategK- 

planning ^.ontexc— so that* for example; institutions seek to exploit 
thcir Lomi:>arative advantages^iversity is likely to be enhanced- 

If there is a tendency to view average behavi<i*fas a desirable 
b.tandard» then more than homogeneity may be irKthe offings 
Homogeneity \\ouId increase, presumably, if instttutions^oWed 
toward either tKe top or the bottom of the scale, rather than the 
middle* By i?^ovmg.toward the middle, or norm, they may simply.be 
entrenching medicK^nty. Certainly they are settling for subop'timum 
performance. The issne is particularly relevant with regard to 
resourLe utilisation. What is 'the virtue of having average student- 

^fjLulty ratios, staff -faculty ratios, and expenditures per student? 
Coordinating-agenty staff, trustees, faculty, and administrators will 

■ differ as to wh^h end of the spectrum is desirable,. Politically, 

, therefore, the muJdle ground may be the safest. It may also sdetn 
appropriate oo more substantive grounds, because of the limits of 
our knowledge regarding cost^ffectiveness in higher education/ 
Because outcomes are not adequately understood, it usually is*dif^' 
ficult, if not imposstbltt^o determine w^iether an institution with 
relatively* low Losts perfetudent is highly efFjfient— or in fact unc)er- 
funded* In the ab&<^nce of definitive 'standards, perhaps average 
belfl^frior does have value as a comparative standard) Nonetheless, 
"being ^average" is not, in and of itself, a goat that will inspire 
innovation and excel^tence. . n 

* Any sort of data may te erroneous or otherVise miSteading, But 
comparative data are particularly vulnerable, for a long list of 
reasons tha^ will be looked at more closely later. We cite here c/nly 
the moit common; comparanve data are often derived from multiple 
SOurLes, the rules for recording the data may be inconsistent across 
sources^ the olose familiarity that Lan be so helpfUl m spotting data • 
errors is usually missing because one typically must depend on 
secondary sources. The severity of the threat to the ij^tegrity of a 
comparative analysis from possible errorwn the data varies in part 

^with the use and purpose of the^comparison. To take one example, 
the threat- is likely to be greater irf a management-cot^trol content, 
tthere the analysis may require highly accurate data to be usefiilf 
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than in a strategic -planning context, ^wh^re data need not be. so* 
accurate (Gorry andScott Morton I97i),^ ' 

FinalK, must consider the (.ost of comparative data^f takes^ 
significant resources to generate, btore, and analyze data. Assuring 
that the^Jata are in a form appropriate for comp'arison purposes 
adds to the ^^st. And solid (comparative studies usually take the tim^ 
and talents of experienced analysts* In hard times, such as the eaHy 
1980s have bcen^ the costs of building the necessary databases and ^ 
(wOnducting <;omparati\e analysis should be accurately weigh^ 
against expected benefi,cs. \ . ^ 




Complexities 



The complexity of higher education is mirrored in the preparation 
aAci'use of comparative data about the enterprise, To begin ivith, 
chokes must be tpade about the unit of analysis^ the particular issue 
about which comparative data will be gathered;' and the type'of 
group that' will be used for comparisons, 

# 

Unit of Analysis 

Essentially, the choice is among four units>The first unit involves 
institutional com^oneriCs— ^ograms, depart ments^olleges within*^ 
unyersity, or other budget oi" acnvity centers within the institution. 
Cost analysis at an institution usually includes this type of internal 
unit-levcl comparison* Less frequently, departments or fit her 
activity units will be compared acrc^s institutions. Such matters ^s 
rcsource'^utiliiation, salanes, workloads, and quality are usually the 
issues in question, » » 

Next most common, we believe, are comparisons between or 
^mongentite institutions {usually confined to single-campus institu- 
tions). Faculty -salary comparisohs clearly top the H&t; they long 
have been rputiniied, principally, through annual studies conducted 
by the American Association of LWversity Professors (AAUP) and 
' the National Center for Education Statistics (NCES), Comparisons 
of tuition rates, costs per student, expenditure patterns (distribution 
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of expenditures by functi^g}, ac^emic programs or services offered, 
and (.luaomes (for instance, degrees awarJed across programs) also 



s 



UK frequentlv made. PeriodKatly, the more prestigious institutions 
re compared on the basis of quality. 
Third, comparisons among states have become the rule rather 
than the exception in recent years* Though pften Surrounded by 
controversy, interstate comparisons of financial support for higher 
education have becq^me integral factors ^ ^he funding process in 
many states. Salaries, resource-utilization p&tterns, costs per 
student, and tuition rates, usually compared annually or bian- 
nualty, are of primary interest* A comparison with other states 
normadly is made lihen state-level officials constltJer major changes 
in .1 funding formula. (These formulas incorporate fixed, quantita- 
tive relationships, usually between number of students and allowable 
faculty. positions, or number of students and^a sugg^ed funding 
level. They are used as a basis for deciding the appropriate level pf 
funding for public institutions in roughfy _,half of the states*) Occa- 
sionally^ a state coordinating* board ot a legislative staff may 
^ompare the structure of its entire system of higher education with 
those of other states* Or it may compare a paiiticular sector, such as 
tie community -college system, with similar ^sectorsMn other 
states— with respecf to unit costs^ accessibility, and so*forth. 

Fourth, comparisons of higheitedutatiori systems 'in different 
countries are made occasionally- They are ujiually conducted by 
aLademics specializing in compg^rative edi^caticfri. The National 
Commission on Excellence in Higher Education was chartered by 
the Congress in 19^ to^make international comparisons* Political 
issues have arisen ifc^SrdiFig some Commission findingSi^uch as the ^ 
fact that the Soviet Unii&n and Japan are ptoducing far more engi- 
neers than is the United States* Historically, such comparisons ^iave 
been potent— witness the comparisons between U.S* and German 
yniv^rsities in the ISt^century, which immensely influenced the 
development and chatacter (jf graduate instruction in this country. 
International cwnparisons are likely ^6 have continuing importance 
in our increasingly competitive yet increasingly intetdependent 
world* 

— — ' • /^ ■ ^, 
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CumparatJvc anolysj^ c^m be applied to any number of manage- 
ment i_o[Kcrns», including tuinufi rotes, salaries, curnculum 
struLtujfc, (.lietttele, workloads,' produetivJtVt^ outcomes, quality, 
unit e\pcnditures> discnbutiun expenditures by function, UQit 
revenue^ distribution of revenues by source, rank arfd te^Sjre 
structure, administrative structure, and governance str^lcturc, A 
complete list would be jridefinuely long. Financial comparisons^ 
aippear to be most frcquenctv-ri^jade— and more often than not, tKey \ 
create conrrovcrsy. All of the issues u^e have listed are often the 
dir^vt foLUsdFa comparison. Instead, they may bejexamined for thV 
idke of properly interpreting the meaning of some other data in th^ 
.analysis* (iTcLhrncally speakings they would then functioning as 
mteri*^nm^, or contott^/, variables,) Issues such as comparative , 
expenditures per ^tudenc across institutions normally oufeht/o be 
looked at in light of comparative data on related characteristics. 
And the proportion of students 'enrolled m*high-cost programs, 
the proportion of graduate students, and^milar data A^ould be 
important factors in interpreting per-studeht expenditures. 



Comparfeon Groups 

In addition tochposinga topicand unit of analysis, chose engaged 
in a'comparative analysisNnilst also choose among several possible 
types of comparison groups, Somecimes the raciorialf for a com- 
parison is that the units are all within a common jurisdiction— for 
example, the departments withia an ipstitution/the institutions 
within a state, or perhaps the institutions withUi an athletic con- * 
ference. Another common reason for a comparison is compettucH^^- 
The level of analysis is most often institutional, since colleges and 
universities have abiding interest in their competitive status vis-aTis"^ 
Rtudents, faculty, and financial resources^ In the case of jurisdic- 
tional groups, the umtsbeing compared may havelirtlein common. 
The units in competitor groups are likely tobesimilarJ>ut they need 
not be; a Vi^all private institution may compete for w>me students 
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uuh a nearby state univetsity* Another cyp^of Lumparison ^roupr * 
uhiLh uill tall iM\ asptrauon group, has tjUite similqj analytical » 
units, Bv dcfjnjcion, however, tht^y differ from the ^:on:iparmg unit in 
at jeaiit.one major regard— that whiLh has OLLasioned the Lom- 
parUoR. The disparity might exist in suLh tthings as fa^:ultv salaries 
or juition. Usually, the 'comparing unit la interested in whether the 
gap should &e closed. A fourth type is a peer group. Pten units iire y 
thought to be essentially similar with respect id contextual factors 
important to a particular analysis, if not mere generally* TKcjr 
overall similarity allows comparative data^^ay on resource ucjbsa- 
tion^ to yield a certain amount of nieaning that they otherwise 
wbuld not have.rcer^roupanatysisis^amined in some detail later, 
be*.auije it is potentially a rich source of significant infortnatton of 
m^ny kinds but is hardest of all to do righr, 
' ^ So far have b^en tonce'rned primarily with Mi,hat is po&sible by ^ 
^Vay of comparisons. The acTmlnistrator must also oDnsider the hou 
aod ufuit /or of comparisons/ For our purposes, fioii^' refers mainly to 
theuay in which the comparison group is developed. The procedure 
selected will depend in part on the typ& t>f comparison group \ 
^ required. For exampfe, a statistical procedure is superfluous if inclu- 
sion in a comparison group is dictated solely on the basis of legal 
boundaries. On the other hand^ one may have to rely on statistical 
procedures to determine the composition of a group of peers. ^ 
What for refers to the i^se a nd purposeof the i ntended comparison. 
• They <ire likely to influence data requirements.^For example, com- 
parative data are occasionally used to generate ntimber values for 
formulas* (Minimum salary levels for various positions atid ranks 
^ma\ be determ^jed by taking a percentage of the mean [average] 
salary le\eU at ^mparable jnstttutions nationally, or regionally, or 
within a state.) FcJr such purposes, the data need tcbe much more 
prcLise than if the comparison is intended to provide background, 
information only. Similarly, a higher level of ^^^^ accuracy isjikely 
to be needed^when the purpose is'assessment of performan^^e as 
opposed tu strategic planning. In short, both desirable data proper- 
ties and data requirements generally are a fianctiortj^ intended use 
and purpose. In turn, these requirements^ay also affect thechoiLe 
of a method for developing the comparison group* * 
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In addKion^ the appri^prtateness of a parti(:ular mechc3^of 
>i.ompansunj in^:iuding the^'develogmeftfof the comparisoa^^pj 
depundslon another sec of^ontexcuarmatcers— charact^rtsticsuf th^ 
intended audience^* policial "tonsideritio^s, and other practical 
conMdcration&. The intende!H audienctMncludes evetijone \i1io is rcf 
receive the comparative report -and perhaps act uporv llQ^Ttie , , 
• ^.redibility of the analysis m^^v be put at s<?me risk if thje an^Iy&is 'is 
not fiilly understood by this emij-e audience. Understanding should • 
/ . ejLtentJ to the general concept of the statist icaJ*m^thcjd*employed, 
and the nature of the data (in the broad sense* of' a^cjrpanting^^ 
. ' rputinest typical assumptions* ^3tid sucti). Possible secondary 
lauduenc^s s^iOuld not be^ 6verI6oJced, Ajeport requested by t}ie 
prcMdent tnay also end up beingreviewed by che^boSrd of cg^fees or , 
. featured in ch^ alumni newsletter! ?Thes? possibilities may >uggest - 
something about the appropriate design fof a compar^tiv^iBr^atysis— 
th? range ot issues^overed undei- a^gpneral copic such'a^ compara^ 
tive srudent oufcome^^ perhaps* or the list ofpeople to be giv^n an^^ 
, opportunity to shape the study in somg^va^; ^ *^ 

When preparing d^ta for analysis of virtually fliny kir)d; it usually , 
pays tot.on&ider t he-enveloping pofttli^! context. Totally (nnocubus 
, cgm'parative^at&'{ff &uch existj araiiot^^orth gathering a^d dissemi^ ^ ' 
^ nacing. U^ful data have the potential to make a difference— someone' 
'may benefit;. another may.be thre|itened by'^eir QS^/Ide|il^j tlie 
intere5ts*of everyone m}h a^take in the-comparisoq should some^ 
how be represenred in the process, This is nearly al^ay^^nattam^ 
' * able, of course^ but the polui^al ramificatigps-need to ,be thbught 
■ ' through carefully. Bein| the rnore political animal?''ttte adminis- 

trjator should assist afVd monitor the analyst in tl\ese matter?. * V ^ 

C^Firijlly, baSjc pracficol questionf remain;^ow mueh will'the 
aj?aly sis cost, what data arc avaiJabl^K^^ ^hai sort^^^apalytitjal ^ * 
capability (peppk^ software, hVdware}areat hand? 'ftese (Questions 
af^ aliVio^t always relevapt when re^eiting studies ofxlata on one*s 

o\^n insticuEional experience*-, (Iri^titutionai rese£[rct\p'ts (^11 them ^ 

empirical 5tu<ife5.j They can bejei^Gialiy important for c^Omparati^ve 
reports^ becau^ those preparing the reports may have to go beyond / 
tbeir customary 3ata base ^d analytical'pr6cedures to get the job 
dor\e.' I ■ ' A 
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, To h'um up, comp^ci.soni arc ,cumple:v analytiLal undcrtaking^j 
With numtTou*^ jnttrrcLitt^ faLCt^. The adrrnnistrat^ir-analyst team 
hj^ lirfle Lhoj«.t.* hut to deal with .sek-Ltiun of* a unit of analyMs, fu*.al 
JS^uc^, JTVp*^of<.oTnparison group, a method of*de\ eloping the Lom- 
pnnson, and some practical uonsi delations, su^h as tost. The other 
Jimensions— the intended use and purposeof the data, the nature of 
the audiences for ihti analytical results, and various political 
matter^— may not aKv aysbe fully and mutually recognized and con- 
sidc^PC'd. ))t'e tjrgJie that they ought (o' be, however. Indeed^ these 
latter aspects may be the tnore jmpblrtan( ones in determjnjng 

er the Lomparatjvoc^data thntlti^e generated go on to become 
mformatjon— that is, data useful to^managemcnt. Figure? 1 sho,ws 
the many dimensions of the comparison prcKress, 

ha\e noted that (he tatk of transforming data intu^nforma- 
iJoti^is the Lentrat theme.of an earlier took in this series, by Dennis 
Jones U982),Thc message tKerc is straightforward. Transforming 
'data into information is'not aucomatL, It takes effort, insight^ and 
expenc^nLC, It typiLatlynCan best be carried^ut b^ a competent infor- 
mation profesMonal^dtSunbed by Jones as having a threefold ability, 
"XD to understand the maoagerpent problem, (2) to appreciate the 
pcrspcLtive from which the W;r addresses the problem*, and (3) to 
idt^ntify and appropriately analyse the dat^ that wfti best inform the 
User Lonfronting the problem** (p. 48)* These abilities are in addition 
to the data-related skills that we usually associate with an analyst. 
The data-versus-iOfdjmation issue becomes more critical, if 
anything, w hep the data ire <^omparative. Thus the present wTiters 
have a twofold objective, Om is to help the administrator ask the 
right questions, formulate'the problem as precisely as possible, aftd,,/ 
'think through the comparatiK^'process^as a whole. This will cut* 
dow n on wasted 6tmc* and effort, decrease the chances of serious 
error, and increase tTie chances of obtaining the^esired^informa- 
tion. Otir sCLond objective is t^ssist the administrator in grooming ' 
competent information profeifeionals, Jones pbserves that ^uch 
people are made, not born. Higher education will be the better as 
they increase in* number and grow in ability. Administrators will 
have much to say about the extent to which th^t happens. 
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The nc^c chapter Considers various uses «f comparative data- It 
Jeal> vs,itb Latcgoncs of use sut^has management tontroK^ersuspIan- 
\\ii\\2,, djffcrent.cs between users (for example, public versus private 
in^titLiTkins)! tvpical uses^ and pitfalls to avoid. Chapter 2 concludes 
with an. extended discussion of financial t^ompansons* \\hich have 
come to be the pnrtcipal use of comparative datg. ' 

Ch<ipter 3 15 devoted to the development of peerewups. have 
seen that various types of comparison groupstan be used m 
generating comparative data. The peer-group option gets special 
attention in this bbokbecauseix is chosen s<ji frequently and because 
It often pushes data to^ or beyond, the 1 1 mi?s~oTac ce pt abl e use. The 
discussion of peer 'group development provides a context for 
claKirating Km prt,Kedural issues, especially politkaljones, that are 
imbedded in any Comparative analysis. 

Chapter 4 includes a discussion of the kinijis^ comparative data 
available and uhat is known about the quality of that data. In 
V LhapTCT 5^ the outlook for data<ollection efforts at the federal and 
state leveL ib surveyed, and likely future data demands ©n institu- 
tion* are assessed. Weconclii"de with some tde^sabu^what adminis- 
trators might do to influence the ftjture use of comparative data. 
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• -Uses -of.. . 
Comparative Data 



Oata are renclered ntither good nor bac^ valid nor invalid,, 
u&eftjl, supctflubus, or dangerous by being put to cdrnparative 
u&es. lE^ (.bri&gjuence, allowable generalisations abo^KjaGmparatae 
data usually arp not really meaningful In thU chapter, therefore, we 
JeV as m'uc h as possible in specifics. Jhe specific u&es of comparative 
vjata that we discuss will be seen, we think, as typical of the more 
common kinds, or categories, of use. / . ^ 



Categories pf Use 



Being held accountable on the basis <jf comparisons, invidious or. 
otherwise, is a fa^t of life for most admi^jistrators in higher educationr 
today. Fortunately, ^comparative data offer opportu nities to turn the 
demand for^ accountability into a two-way street paved with poten- 
tial advantages* V ' ' ^ ' 

£xternal pressure for accountabij^ty comes from two sources; 
governing agencies anc^ fiand^. In American higher education, 
governing boards ©f one kind or another are ubiquitous. They set 
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the- basic policies that govern the institution and select antj evaluate 
the Lhief executive officer* Some of their basic concerns- can be 
addreit^ed with comparative data* Governing boards are concerned 
to know, for example, whether the teachfing and administrative staff 
are being adequately compensated* The question often is addressed 
by reference to rates at comparable institutions (adjusted, as 
necessary for cost-oMiving differences)* Are the institution s 
student^ developing as desired? Again, the achievements of com- 
parable students at other institutions are relevant* Similarly, such 
questions as whether the institution is receiving i^s fair share of 
resources from funders, wh^her it^ i? charging the appropriate 
amounts for servfces, whether it is spending too much on athletics, 
whether it is too liberal^ about th^use of alcohol on the campus or 
too conservative about providing birth-control services at the 
student health center— all sucTi Issues can be more fully illuminated 
by data on practices ^t other fnstiturions* 

The authority of governing boards td require accountability in- 
heres in their responsibility to guard the well-being of the institution 
and ensure th^t it pursues its mission with alt due effort. Funders 
would seem to have at least equal poW^r to demand accountability, 
in the public sector, indeed, the power to withhold funding in the 
absence of satisfactory accountabili^ is theoretically the 'power tq 
destroy* In practice, funders of all sorts and persuasions generally 
behave with marked circumspection, even when the accountability 
they seek elui^ them* Funders vaty greatly, however, in the amount 
and kinds of accountability they w^nt* State governments often 
demand a great deal ^^[accountability information. But usually their 
requests^are cStifined tonnancial accounting— were the funds spent 
as planned?— rather than performance accounting— were the objec- 
tives met? Federal accountability demands tend to center on institu- 
tional performance with respect to the maze of issues subsumed By 
such umbrella terms as affirmative action, civil fights, and national 
security. Private fiipdlng, i^Muding individual philanthropy as well 
as foundatidn giving» often comes with no strings attached. AH of 
this notwithstanding, questions about the cost '.of college and 
university operations and the benefits derived frbm the use of 
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cesources arc in the a^r mote and more. E^cause optimum perfor- 
mance lev^s are so difficult to pin down, daca on perfotmance at 
comparable institutions may well provide the only practical 
fran\ework for assessment. * , * 

Understandably, the adj^mistrator may view as burdensome the 
perennial gathering of comparative data wanted* by a governing 
board and funders. Yet this very data may be, in the same contexts, 
a weapon of advocacy. Comparative data can underscore the rieed 
fot a new program, or additional staff, or higher salaries, or a greater 
share of available resources. How better tobuttress an argument for a 
fair share of resources than to compare services rendered? Compari- 
sons need not be confined ro higher education* Rising salary rates 
foe comparable positions Tn the surrounding city or region can be 
useful advocacy data. Relatively easy position matches caA be made 
for clerical and maintenance staff and for some technical specialists, 
such as compater'programmers. And for some faculty, particularly 
those in the professional schools, real-world counterparts can be 
found^for salary comparisons that have advocative powcjr. In the 
early 1980s, engineering schools have drawn considerable attention 
to their difficulry m keeping faculty by comparing what they can pay 
to engineering salaries in h^h-technology industry* 

In general, the same sorts of questi^s asked by external groups 
are i;elevant to the administrators internal responsibilities. Regard- 
ing outcomes, fgr example, one might ask. Do our students do as 
well on ]i(.ensure examinations as students from other instituticjns in 
out state? Do other liberal-arts colleges award a similar proportion of 
degrees, in career oriented programs? Regarding expeditious use of 
resour^ies, the questions might be: Can our attrition rate for fresh- 
men be considered normal? Is out proportion of tenured faculty iu 
^ tile physical sciences extraordinalry? Are our costs per square foot 
foXphysical'plant rnaintenance in line *ith costs at comparable 
institutions? Any administrator could ^dd any number of such 
questions, to the^list^'^othout promjDfing. In eacK case, the data 
needed for the assessment will necessarily be comparative in nature. 

Comparative data remain useful wlien tHe administrator's 
perspective shifts (ever so slightly) from accountability to control. 
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Management Lontrol js exercised to ensure that policies are carried 
out ^nd goals are met. At the state level, controKusuatly takes the 
form of (Coordination, vtith its attendant reporting requirements. 
The .<;tate will have targets^ although oftei? only implicitly estab- 
libhed,for student acce^, choice, arid opportunity, and also for out^ 
Lomes. Trained manpower ha& long been the primary outcomes 
concern at fh^^ate te\'el, but interest lO educational quality i& 
increasing. To monitor {progress toward these ends, coordinating 
, boards gatTier a consideYable amount of data from the institutions 
vtithin thei^ purview. (In some states, this includes private as well as 
public institutions.) The data bear on questions hk'e these: Is access 
to higher 'education in geographic balance across the state? How 
s>ignifiLant is the differentiation among types of institutions, when 
parental income and other socioeconomic cTiaracteristics of 
students are compared? What is the*difference in the net price paid 
by studer|t& in xhe public sector as compared to the private sector, or 
in research univer sit leiias, compared-^ community colleges? How 
much duplication of effort exists among the states colleges and 
universities?' A re the differences in program quality at various insti- 
tutions sufficient to justify cost differences? 

State-level control may entail the collection of compa^3^tive data 
from othe^ states as well. Faculty compensation rates, for instance, 
may.be adjusted periodically on the basis of such comparist^hs. 
Some states-express in comparative terms their policies regarding 
compensation^and other control factors— "Weshould rank near the 
top" or "We should stay within 10 percent of the average in our 
regibnJ' The state of Florida has formulated various aspects of its 
iong-range pbns for higher educatioi^ in terms of interstate rank- 
ings. In Kentucky, tuition at public institutions is indexed to charges 
at Lomparable institutions in other states (Viehland, Kaufman} and 
Krauth 1982).^ 

Comparative data finds many uses in nl^nagcment cbntrol at the 
institutional level, and even at the departmentallevel. The example 
of tuition price comes immediately to mind because, particularly at 
private institutions, it has far-reaching implications for enrollment, 
student characteristics, institLrtiOttal image, quality of induction 
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and services, and many aspects of resource allotation— all in addi- 
tion to financial heakh. Private in^utions^ especially* k^ep in- 
formed about tuition levels and trends at Comparable institutions, 
and in fact may have a policy to keep their tuition somewhere within 
the range charged bycompetinginstitutiorfe. Comparative cost data 
have aUo been shown to be valued highly for management<ontrol 
purposes (Collier 1980). 

At the department level, informal comp^FJSons certainly exert 
strong influence on an assortment of decisions about academic 
policies and management -control issues, ^'hat oth^r departments^ 
are doing regarding course requirements for student majors, inter- 
disciplinary and double majors, grade and performance require- 
ment! fyr honors pro-ams— all such decisions are taken in full light 
of comparative practices within the institution. At the graduate 
level, comparisons are more likely to involve departments at other 
institutions. This has implications for top-level admimstratiorit of 
course; The chairman of physics may argue to the dean of arts and. 
sciences that comparative data show a strong connection l^tween 
faculty quality and a department's ability to attract feder^ research 
fiands, which in turn facilitate faculty-building. The dean knows 
full \vell that the institution s reputation and image depend to an 
important extent on ^he reputation of its departments^ and that 
departmental reputations ar^ comparative matters. 

We hav g: e&c hewed general izations^ but it does se^ that com- 
parative data are most, and most often^ useful iri decision areas 
where policy and control considerations overlap- Since policy and 
contK)! responsibilities overlap in the persons of hearl^ll adminis- 
trators>and since nearly all would concede the virtual impo^ibility 
of exercising these responsibilities in isolation fro^ln one another^ 
comparative data are Got likely to influence solely a policy decision 
or solely a control action /Plain ly^ policymakers and managers at 
whatever level— state, institution^ or department— need to know 
what's going on around them* How are others playing the game? 
How are other institutions interpreting "financial exigency** in con- 
nection with faculty 'tenure policies? How closely are other stat^ 
linking salary increases to changes in the cost of living? How many 
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i.tate^> Kne imposed eTirollment caps on some* or all, of their mstitiJ\ 
tions? And what impact have enrollment caps had on statewide 
enuillmcnt totals and student migration? How ^do comparable insti- 
tution:^ handle student health care, in terms of range of services and 
quality^ Vt'hat doofhc^r institutions do about sabbaticals? This small 
i^amplc of possible comparisons tempts us toward anpther generali- 
zatLon— that in higher e^^ucation, much com|>arativedata will be of * 
interest at all levels of administrative responsibility, horr^ the 
academic department to the statewide coordinating agency. What 
one physics department seeks on the basis of comparative data, 
phvsics 'departments in other 'institutions within^ the state may 
demand, the cumulative effect canbeaspalp^ibleat the state level as 
it IS at the dean*s level inea^lh of the institutions involved. And few 
statc-le\el decisions about highereducation are so abstract that they . 
escape notice sft the department level. 

Ultimately, we realize ^ policy must be established on the basis of 
mission and available resources. Nevertheless, knowledge of what 
others are doing helps to establish the range of alternatives. 
Sometimes^ if the data are available, onecag foresee with good accu*^ 
^acy wh^it would be the results of various possible alternative 
policies. In other ways-as well, comparative data can suggest what 
might result. from a poticy change* For instance^ suppose that an 
institution IS contemplating an upward shift in its admissions 
requirements. Common sense insists uponsomeeffor^ to predict the 
efifect of the new policy on enrollment. Surely it will be valuable to 
know the status of requirements at the other institutions in the 
recruitment area and their enrollments. If practical, u would be\ 
good to know the effect over time of a similar change in admissions 
requirements at similar institutions. 

Planning, perhaps even more than policymaking per se, de^nds 
on a flow of comparative data* This is especially true when planning 
IS strategic in nature— that is^ when planning focuses on what the 
entity (state system, institution, department) should be in the 
future^ what services it will provide, what niche in the world it will 
try to occupy. To be sure, certain first-order questions have to be 
answered without reference to data on the behavior of Qthers. In 
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particular, an organization needs to know what it could do within 
' whatever immutable constraints are imposed on it, what its 
members are witling to do, and* in the mosc fundamental sense, 
what It ought to do. But the world in which a college; or university 
muit operate i$ ^competitive anojtjircrrelated. However worthwhile 
intrinsically, preferred activities cannot be sustained for long if there 
is no^cmand foe them. 

An axiom of the present time in higher education holds that an 
institutions' key to survivaKagd eventual prosperity) is identifying 
it& uniqueness. VlTiat are its competitiveadvantages? Comparative 
data help both to identify the advantages anc^to describe them 
concretely, so that theV will be understood in an operational 
framework* Institutional goals and priorities can have inspirational 
value, even if theV are impractical. But if they are to be used as 
actual operational guideposts, then they must be expressible in clear 
and* concrete terms. Comparative data can help supply that 
concreteness. ^ ' 

Projections, estimates, and forecasts of one kind or another are 
basic to planning. All can draw support from comparative data. The 
likely success of a new program, or the continuecJ success of an exist- 
ing pr^^gram, may depend on the availability of comparable pro- 
grams at institutions within a particular region. The likely ability of 
an in&titutio^ to maintain viable enrollment c^n be estimated on 
the basis of its drawing ppwer yis-a-vis that of its competitors, (See^ 
Rowse and Wing 1982 for a good discussion of the relative drawing 
power of institutions,) Knowing results of efforts at comparable 
institutions to r^ise funds for Capital investment can usefully inform . 
the establishment of realistic goals fot an institution's own capital- 
funds campaign* * 

Pricing decisions, as we have noted, are often grounded in com^ 
parative data* They are a primary means for deter mining what con- 
stitqtes reasonable charges^ for one thing* The freedom of public 
institutions to determine tuition prices is a variable, and seldom 
corr^parable to that of private institutions^ Nonet heless^ institutions 
in the public sector have a stake in influencing tuition policies and 
lev eb, which comparative data can help them do< Public institutions 
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ako have a voice instate-level decisions about funding forrrru Us and 
appropriations to higher education^ in both instances, a price for 
institutional services is being determined? Tht* bottorn line of a 
request budget is a pn^e that the institu(noh hopes to collect for the 
assortment of services it proposes to pnoyide. State govei^ments 
periodically consider, in all of its poetical ramifications, a ^srt of 
pncing decisions for higher education— what Jtuition to charge 
'resident students as Opposed to nonresident students^ and whether 
^or not to differentiate tuition by level of instruction or type of 
program* The states are likely to seek normative (averaged) i^ata to 
support thetr decisiQnmakit^, while private institutions prefer data 
on competitors. The data a^ comparative in eitjier case, of course, 

ir . 

^Differences Among Users ^ ■ 



There can be subst^tive differences, then, in the way that dif- 
ferent types of institutions use comparative data, even when 
addressing the same sort of issue. The public -private distinction, in 
particular, npples across all the categories of use discussed 5fbove, 
Public institutions are held accountable by both the state and 
students, but more regularly and more thoroughly by the state. At 
public institutions, therefore, comparative data generally will focus 
onleveUof activity, efficiency (jf operation, and outcomes relative to 
stated purposes. At most private institutions, studenis have greater 
ac*.ount ability leverage, b*cause their tuition payments accojfint for 
a greater proportion of operating revenues. Thus comp^fattve data 
^interest to private institutions will focus more on net price, tost of 
recruittnent, attrition, and outcomes relative to student aspiratiQns 
(job placement, graduate- and professional-schoo! placement, and 
so forth). Also, privatepstitutions are quite sensitive ^o the views of 
alumnf and private amders, and may have occasions to Cise com- 
parative data that speak to their particular concerns-iwhich 
sometimes are at odds with the concerns of current students. 

Differences in institutional independence also af5fect data require- 
ment&^The locus policymaking, and planning tends to be 
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mt^-Tnal private institutions. It is shifted somewhat toward the 
intcrnal-vxternal margin at pjjblic institutions— with considerable 
vdfi^jtion from state toJ^tate. Consequently, comparative datii prob- 
abl> ^jre moht often used by public institutions for justification and 
* advocacy. At private institutions, on the othef hand, comparative 
data are more often used in analyses of competitor practices. But 
when private institutions seek state assistance, thef^too use com- 
parttrue data as an advocacy tool, in the fashion of their public- 
sector Lounterparts. For example, the share of statewide enrollment 
Carried by the private sector and its contributions to manpower 
development are likely to be emphasized in putting forward a case 
for assistance. 

One difference between public'and pnvateinstitutions that seems 
let^s important than one might expect has to do with confidentiality. 
While data in the public sector are perforce public property, the 
same cannot be said for data in the private sector. But the confiden- 
tiality of data in the private sector appears to impede c^flfnparative 
analysis only rarely. Voluntary participation by private institutions 
in the Higher Education General Information Survey (HEGA), a 
majoi* source of comparative data, is quite high. On occasion, 
though, especially in a few highly competitive urban areas, data ex- 

^change among prwate institutions is deliberately limited. And some 
data collected in the public sector may in practice be hard to obtain. 
Trying to compare the full costs of athlfetic programs at major 
universities, for instance, surely would be tremendously difficuh-^ 
and not just because of technical problems of analysis. 
Still other differences among institutions cause variations in the 

'"use of comparative data. Major research universities, whether 
public or private^ are so laige and so complex that overall institu* 
tional comparisons are meaningful with respect tooTvly afew factors, 
at -best* More ^?ften than not in this sector of higher education, the 
comparisens of interest are those at the unit level— comparisons of 
undergraduate cpUeges, or law schools, or English departments. 
Data are available fRSt allow cornparisons of depSrtments at well- 
known institutions' on the basis of quality and effectiveness. (See 
Lawrence anH Green 1980 for an evaluation of all but rhe most 
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recent of these comparisons J Smaller, more narrowly fucubed ir\3ti' 
tutioni; find more utility in overall institutional compari^nb. Topics 
^uiwh :>p'Jnsored-re^arch expenditures per faculty member inter- 
etrt only a limiced set of smatir institutions. Normative data on 
remuneration for teai^hingassisCance is relevant to a u ider, but still 
limited, set of institutions, ' 

The environment (.an make a^much difference as institutional 
(.harj^-tcristii^ in the^ay comparative data are used, A community 
(.otlege in'a remote region where there is no other institution of 
higher education ordinarily will have' data needs rnui:h different 
from thoNC at an urban community joltege competing for a portion 
of its students w ith a variety of alterna?i,ve providers. We said earlier 
that some situations are so competitive that data*exchange can be 
inhibited, in such free-market envii^onments, trade secrets are pro- 
tected and data exchange is viewecf as a threat,. We may see more of 
this in the future, as more^ institutions begin to encroach on what 
was previously someone elses territory. Public institutions will work 
harder at recruiting and at securing private' gifts, Liberal-arts 
^.olleges will seek out the sCKalled nontraditional student and offer 
mu4.h more in the way of occupational or professional programs. 
Paradoxically, even as competition increases and the value of com- 
parati\edata is enhanced, the data itself mavbecome more difficult 
to obtain, V. 

One more environmental aspect bears mentioning. Many public 
^.ampuses are part of a system of institutions. In this circumstance^ 
system offit.es make systetp-leve! comparisons for funder-required 
reporting and for management control. On occasion, system ofifices 
will speak for all the component carTlpuses in advocating additional 
resources or new prograins. This arrangement contrasts sharply 
wuh the independent institutions that speakdirectly for themselves 
to flinders* Of course, institution^ operating in a syscem can use 
comparative daca to protect the status quo or to advocate their own 
enhancement when negotiating with the system office. However, 
one would expect the range of data^allowed as **evidence" to be 
somewhat circumscribed by system-related constraints. 
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TypicaTUses , * 

Up tut hi^ point, have surveyed ;> wide r^geofpossibleijtcsfor 
Lomparan\e data. &ut \^hat sort of data are actually being used fa^^ 
Lomp^jrativtrpurposcs, and by whorti? Unfortunately^ fhere has not 
yet bgpn a uompreheiasive national study pf this subject. However^ 
tliere have been studiqs focusing on the use of data collected in the 
Higher Education Genersrl Information Surveys (MEGIS). The 
HEGIS 5ur\eys are discussed in greater detail in chapter 4; they are 
part of ongoing efforts by the National Center for Education 

, Statistics to gather a variet^^of data on the entire population of 
American colleges and universities, whicKindud^s some 3,3004nsti- 
tuuonb. HEGIS constitutes the broadest source of comparative data 
now available* How it b used should be a good, although not com- 
prehensive, indicator of how comparative data are being employed 

teener ally. 

The most thorough sfudy of HEGIS data was completed in 1980 
(Andrew, Fortune^ and McCluskey), Both .riistitutions antj state 
agencies were surveyed regarding the types of HEGIS data used^ and 
why. For our, purposes, a short summary of tlhe results will suffice: 

K The most widely used data deal With^enr^lment, degrees 
awarded^ finances, ^nd facul^ salaries^ in that ordef— each 
type'being used by more than 50 percerlt of the respondents. 

2, The following types of data were used for making^mparisons 
across ir^titutions {numbers in parenthesis show percent of 
respondents that did so on one or more occasions): faculty 
salaries (49), enrollment by discipline (48)> degrees awarded by 
discipline {45),enrollment by sex (44), degc^^es ^warded by level 
(43)> enrollment by race (40)> finan^^ial starus (40), degrees 
awarded b^ race (39)> library quality (38)> pi*op<3ftiorl of faculty 
tenured (38), residence and migration of s!udenA 06), degrees 
awarded by sex (35)> and classified-employee sdUries (31). ^ 

3. ^The issues regarding which HEGIS d^ra were most^quently 

used included affirmative action^ piygrams, costs perstu^enit 
mix of students, and institutional status (finances, library^ and 
* so forth). X * * 



36 



"31 

J ■ 



CHAPTER 2 



A primary use of HEGIS data is to support policymaking at the 
natjonal level. Chief concerns inLlude enroltttier^t projections, the 
statu*! of higher education by scLtor (private ani^ public), manpower 
planning, the finaHLial condition of higher eduLationifacihties plan- 
ning, and libr^try planning, ^'hile the sphere of institutional admin- 
i^strators does not embrace national policymaking, their institutions 
do support that poliLymakingby providing data— the samedatai as 
It turns out, that make possible many comparisons at the state and 
institutional levels. 

Partly in response to needs expressed by partic ipants m the stucly 
of u^es of HEGIS data, NCHEMS established tn 1980 aservice that, 
makes comparative data readily available to the Kigher-education 
community, The accumulation of requests for this service, while by 
no means constituting a systematic sampling, does provide a fair em- 
pirical ba&is for dra\^ing a few additional inferences about typical 
uses (and users) of comparative data* Since most of the dajt^ pro- 
vided by NCHEMS derive^from HEGIS, the service record may be 
vie\\ed ?s a kind of addendum to the large study we have reviewed* 

At this writing, the NCHEMS Information Service has generated 
about 1 ,500 reports on compararive data for about 150 individual 
clients. Three conclusions can be drawn* First, interest in financial 
data predominates. About three-quarters of the reports gen^ted 
ha\e been in standard formats— that is, preprogrammed and thus 
easily identified in terms of overriding theme or topic. Fully 60 per- 
cent of the standard reports requested have dealt with finances. 
Another 15 percent have concerned (acuity salaries. The most 
frequently requested financial reports have dealt with revenues {by 
&ource) per student, expenditures {by function) per student, percent 
^f revenues by source, and p^cent of expenditures by fianction* 
Twelve percent of the standard reports have drawn on enrollment 
data and 10 percent on degrees-awarded data. Most of the reports , 
.nave displayed comparative data for single institutions, rather than 
aggregate data by institutional type (which also is available)* 

Granted the preponderance of interest in financial data, it is 
apparent that the primary uses of comparisons have been either 
budget analysts and financial ptanning(takingboth terms in a broad 
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sense) or what one might call finan^:ial advocacy, tn which compara^ 
tive data *ire used to argute for a change in fianding,or allocation 
leveU. 

0\erall, Ldmparison of information-service requests during the 
early 1980s with the uses of HEGIS data m the late 1970s (as 
reported by Andrew et aL I1980I) indicates that the use of com- 
parative d^ta to anaTyie resource-utilization* patterns is increasing, 
A lesser but still substantial interest in structural issues— proportion 
of effort by instructional level, outcomes, and su^: h— is evident: ed in 
the number pf requests for enrollment and degrees-earned data. In 
the custom reports prepared by NCHEMS, enrollment datSi are 
requested about as often as financial data. 

Many other issuer are no doubt being addressed with comparative 
data. A particularly creative use of such data was accomplished by 
staff at a state agency. They combined an index of institutional selec- 
tivity uith listings of the outof-state institutions to whi<;h their 
students had migrated. Together^ these data sets gave a good indica- 
tion of how many talented students had chosen to leave the state 
*and the types of institutions that attracted them. 

So f^r, about three-quarters of the organizations requesting xrofn- 
paratue data from NCHEMS h ave been colleges and universities; 
the rest have been state agencies, re&eafch organizations, or private 
consultants^ The institutions have been about evenly split between 
the public and private sectors, and between those ^with larget and 
those with i>mall enrollments. Most usejs have^been eithet acfmihts- 
frators (presidents, deans^ financial officers^ planning officers), 
administrative assistants, directors' of admissions or personnel^ or 
technical staff (institutional researchers, controllers)*. 



Financial Comparisons 



Financial comparisons at the institutional level &eem as controver- 
sial as they are popular. In principle^ they are no different from com- 
paris*jns ^long various other dimensions of institutional behavior or 
structure. In praqtice^ though, they often are questioned, and witlv 
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govKl ri^asun. Fmanaal comparisons dentve consideration here at 
somo k'ngth. 

The mam point of many financial Lompansons is relative 
(L*Lonomv o(\.obty* Can some insntutions do a similar job for less 
money than othersrDoi,(!j^c in&titu^onsget more for th^ir money? 
(5ther Lotrynon financial questions relate to ihe fair-share principle 
in budgeting. Are the faculty receiving their fair share of institu- 
tional resourLCs? Is the instructional function getting its sHare? Is too 
mu^h money being spent un Lentral administration? How about the 
library? Another set of questions concerris sources of revenue^ 
balance sheets and financial Ratios, and prices charged {tuitijpnsand 
fees primarily^ room and board to a much lesser extent). And still 
more issues l an be addressed under the general heading of financial 
Lumpanson- There are sufficient commonalities amorig them so that 
the following commentary can be taken to relate fairly closely to all. 
Important exceptions will be noted. 

Three LonJitiont must be mtt to ensure that a financial compari- 
son b not potentially [jiisleaclirtg. Firsj, the products and services 
g^^netateJ by the institutions in the analysis must be fully known to 
those making the comparison. Second^ all institutions in the 
analysis must use similar data-^recording and accounting practices, 
or any di&crepanLies must be known so that the appropriate adjust-^ 
menU can be made. Third, any relevant environmental differences 
affecting^the institutions compared must be known and taken into 
account. 

If these conditions are met, interpreting comparative financial 
data IS straightforward, the data can be taken p^retty much at face 
value* In most instances, unfortunately, either the conditions are 
not met or considerable uncertainty exists about the actual state of 
affairs at the institutions in the comparison group* 

Consider the problem of proclticts and services. The mix provided 
at even a relatively small institution can be sizable and complex. An 
exhaustive Knowledge of the institutions in the analysis is not re- 
qgired* Bu,t even a level of knowledge sufficient for the purposes of 
the firranual comparison may be dtfficultlto obtain with affordable 
effort. An abundance of evidence indicates that marked differences 
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m prugram miN, student mix Oevel^ status, abilK>), breadth of ^urrk- 
ulum^ the Tiiiigc uf student bt*r\iLes, ond'phvsiLal-plant operations 
aro the VuU* rather than the c\Lcption, c\cn among institutions of 
the same basn. type, Tliesc diffefopu^affet^t jx^r-^student expendi- 
tures and the pruf^ortion of expenditure's going to various funt^tions 
and activities. The resulting finan^^iat variations among institutions 
that ^ho^^ up in the ^omfSarison may have nothing to do with 
jeljtive economy or ii>thfiv performance; factors. So the data cannot 
be taken at fat.c val^%-flnd interpretations must be tapered, ^ 

The condition that data-recording and accounting practices be 
consistent uin be violated in several ways. To begin with, inherent 
difficu^ies arise in the simplest counts^ such as the number of* 
students, the nu m^ r of faculty, and the number of programs. The 
number of students is basic to many financial comparisons, as a 
means of putting ^penditures orTevenues on a unit basis. Institu- 
t^ns vary^c^^nMclerably, however, in the way they count ^otnts^ 
esp<;c^atly >Mth respvxt to full-time equivalency (so do t\)^ sWes^ see 
Rhodes and Temple 1976). Irv addition, the fall-semester count, 
which in a general^ way functions as a benchmark in higher educa- 
tion. Will not necessarily be equally representative across institu- 
, tions of the true annual activity level that annual financiaP^tistics 
^ ' are ordinarily sugpdsed to reflect. f^"*' ' / » 

Determirungthe number of faculty is not always stfaightforwatd — ^ 
either- To someone unfamjtiar with higher education^ it might seem 
strange that ^university w^ould resort to rather arbitrary rules in 
determining how many faculty it has at a particular time. T*he 
mystery disappears when onie considers the myriad ways in which 
students receive instruction or related services— formal classes^ labs, 
research projects, dissertations, and so forth. The faculty are ajj^tlle 
hl&art ofran ahnost impenetrably complicated process. It consists of 
all manner of &o<aIled )oint inputs, invotvijng students in various ' 
\ roles, interlocke*^ teaching and research efforts, servicers jointly 
generated and cJtisumed on andoff campus, anjd soon. Unraveling 
^ that process for accounting purposes can-xb/ more than difficult. 
Potential problems with these most basic counts are part of the 
reason it is so hard to obtain a goo^ match across institutions for * 
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finariLial Lomparisons, This is usually more true at com^Drehensive 
tiafvcrsitie^ than at smaller institutions. In {^m, a lengthy study 
cv^jrdjnated by NCHEMS led to the LonLlusion that some Lompar- 
jl-jibt>'problL*m^ involxffT^g mjjur research universities may simply 
be unresolvablc (Tt^ing 1979), 

ProspuLts for standardizing a^LOunting practices were cor^ider- 
ably enhanced^tn 1975 when the National Association of College 
and University fiysincss Officets (NACUBO), tlie American Insti- 
tutc of Certified Public Accountants (AlCPA"), and the Nanonal 
Center for Higher Education Management Systems (NCHEMS) 
came to agreement on a vayety of accounting issues> Studies under- 
taken subsequently to assess the quality of HEGIS financial d^ja 
revealed, hu>5tc\cr, that widespread discrepancies in data -reporting 
fjrocedutes were continuing (Minter and Conger 1 979a tb,c),^ Con- 
forming to'' national <{ccounting practices is voluntary* and it may 
LunfliLt v^Jth d system that an institution has successfully ernployed 
over a long perioJ^jn in some instances* with a system .mandated by 
the state. •Conf9rmity^eea^fe to be Improving slow!y> 
* For puGlietv controlled fdstitutions, the accouriting problert can 
be ftirther Lomplicated By \anations in the way in which state per- 
sonnel systems are organized. For example, soW states have a cen- 
trally administered, separately funded pension fund^or all state, 
employees, A fiarticular institution s share of these expenditures can 
Be i_alculated, Bl^ at th^ minimum, it requires an additional sifef) 
that may or Jnay not be included in the preparation of tjie institu- 
tion's cCistomary ftnan<;ial.rep6rts> 

Differences in institutional environment can influence the mean- 
irig,pf financial data- The most direct^fec omeS from differences in 
tijc cost of living and regional i^' age and salary levels. These factors 
ought to bf considered when comparing unit costs,^unit revenues, 
and Lampei^tion,3ut this sometimes difficult to do. For instance^ 
cost-of'hvingdata can be imprecise with re^Pect to an institution nbt 
loLated in one of the 40 major metropolitan areas in^^ded in the 
Bureau of Labor analysis. Other environmental effects are not so 
obvious. At urb^n institutionsAhe category "Operation and Main- 
tenanLe ofPlant"mav cover a range of services significantly different 
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froTn those^reported under the same rubric ty rural institution&. 
Important differences in s^:heduimgand cli^ssroom-utili^atiotioi^tesj 
cipeciatlv in the evenings, may not be revealed in readily available 
comparative*'data» though they ought to be taken in account. 

Some analysts extend envir^snmental ^:onsiderations mu^:h 
fiarther. State or regional ^:haracteristics^ in parucula^ are 
somelimes used in the early stages of developing (:omparison groups 
ptee Dunham and Carter l975j Teeter, Rawsbrv, and Hoyt 1982). 
Srate wealth, the degree of urbanisation and industrialization^- 
pppulatton^ ancf &(^orthj (:learly are relevantto comparisons of st^lte 
systems. Their val% when comparing individual institiKions is not 
so obvious. If the institutional data at hand are inadequate^ srate 
'ch^iracteristics may serve as proxies for ^hat is missing. But with' 
adequate data on what the institution dijes from year to year, addf 
tional environmental data may be beside the point. Take the case of 
two public institutions that are well matched, except that one is in a ^ 
poor state and thaother is ina relatively wealthy stare. Th^fchances 
are great that the rkh-state institution will have higher unit 
revenues (a ^ piHlCmti res) than.the other. If the main concern in 
the comparison is relative costs, the greater economy displayed by 
rhe poor-srate institution should notbe discounted simply because it 
had fewer flnan*;^! resources at its disposal. In oth^ words, explain- 
ing a difference int^sts is not rhe same as explainin^^it away. (This 
line of thooght may be pursued in Bowen 1980j Specially in fcis 
sections on the "revenue theory of cost.*') 

To summarise, the validity of aXinancial comparison aipong insti' 
tu;i*>ns can be threatened on thr^sid^hidden difference^ in the 
products and services of the in&tjfuJteR^^ disparities in ^eir data- 
recording and accounting pra^:tices, and differences in the environ- 
ment within which the institutions operate.' These threats suggest 
thar it mighr^e wise^st to forego-financial comparisons. Perhaps. 
Yet as'we observed at the Oursetj there are good, even compelling 
reasons for most institutions to make such comparisons. So thebest^ 
course usually is to undertake financial comparisons in a way that 
maximizes rheir utility ^nd minimizes ^he threats to their integrity 
and v^dity. The following check list may Be helpfial. 

; 
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L Take the prcMiess seriously. Put knowledgeable people to work 
on it. Do some prior thinking and planning; carefially deter- 
mine* the subject and use of the prdjected comparison, and 
decide how much imprecision can be tolerated in the data. 
Take political considerations into account when determining 
who should be involved in the comparison, 

2, Acknowledge potential p'roblems and threats to the validity of 
the comparison. Even a relatively minor problefn can cause 
considerable harm if it is not take^into account at the outset. 
And proper problem recognition is the best assurance that 
those comparisons that ought to be abandoned will be, and 
that the rest wilt go ibrward on an acceptable basis, 

3, Under virtually all circurrfetances, comparative financial data 
should be interpreted (Xmsewatt^ly. Which is to say, that such 
data should be thought of as general indicators rather than 
strict measures, * ' 

4^ Use a multivariate apptoach,The mote that financial data are 
considered in isolation from other facts^ the greatet the 
oMtnces that those data will be setiously misintetpteted, 
Gathet a sufficient amount of additional contextual data about 
the institutions in the analysis; so that theit petformance on a 
■given financial dimension can be meaningfully interpreted, 

5, The financial data in question should be subject^ to rigorous 
yetification ptocedures in certain situations. The moteimpor- 
tant the data are to the comparison, the greater the need (or 
justification, Bey6nd that, what do the data theffeelves 
suggest? Unexpected variation"& from one iristitution to thfe 
next and odd -looking values or distributions suggest a need to 
find out mote about th^ data. 



State Comparisons 



A^ong the most consistently coijttovetsial financial compatisons 
are tntse done at the state level ^analysis. Few institutions ate 
ditectly engaged in such/comparisoos. But public-secfDt institutions 
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in parnc.'jt^i' (.an be affected by comparatue analyses unJerpken by 
(wOurJinjtingJbuurJsJcgislatHc staffs, regpnt^*staffs^ or independent 
analysts. Thus a fevv comments on statewide financial compansons 
arc in urdcr htTt\ injinc with the general principles we have anun-^ 
^^^ated. Fir^^tj (.ompansons among states are potentially as hard to do 
correctly as comparisons' among institutions. There is m less corn' 
plexitV in handling the three potential trouble spots— products aod / 
services, data-reportmg and accounting practices, and environ- 
mental differences. ^ ^ . 

Actually, the differences *n products and services, or r6le and 
scope, among state systems of higher education can create more 
complications than usually are encountered in institutional com- 
parist^ns. State-level analysis must be concerned not only about the 
large components in the respective state systems but also about the 
composition of these components. For example, a sta^te system may 
embrace se\eral institutions, some with single campuses atjd some . 
with se\eral. MbreOver, &ignificant,diff<rfences in institutional role 
and scppe may be present in each ofthestatesystems included in the 
comparison* Policy differences across the states, such as th^ varying 
extent to^^^iwefe they fund hlghft education through student aid 
rather than institutional assistance, can also complicate compari- 
sons between stat^ systems. So interstate comparisons can entail all ^ v 
of the problems found at the level of institutional comparisons, btit 
\yith the complicSftpns multiplied several tim^s^ver, ' ^' 

When coftiggrin^ thstltutions, the basic accounting, issue is ^o 
determinainstan'ces in which'adrliinistrators at a particular institu* 
tton have elected'to depart^from standard accounting practices. 
When comparing &tateS',ithe, basic concern is to catch diff'srences-in 
the^ay stites provide funds foi^igher education, Some^ states 
include <Uition rexenue in total state appropr rations, spme do not, 
some do, and some do not, allqw institutions to keep recovtoed 
j indirect costs. Many such differences must be looked for 

Contextual \ariables are especially important.w hen state compar- 
isont are^made. Ranking^tdtee by per-capita appropriatiohs for 
higher education is'a case iff point. Tliese figures should be inter- 
preted in light of suElTvariable^ as each states relative tax effort and 
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tax Lapacit>, cust of living, and independent higher-educajion 
resource. Tht^se come readJ> tu mind. But less'obvious differences 
among Uates, such as the age structure of the pcjpulahon and other 
demographic LharactenstR^, at^o might apprupnatelv affect inter- 
pretation of the significance of a^ states rank ^ah regard to per- 
capita appropriations for higher education (Lingenfelter 1982), 

Comparative studies that focus on overall state appropriaticttis to 
high^F education are a likely source of misinformation, especially if a 
lot of precision is needed or expected. States can support higher 
education in numerous ways, and it is difficult to find data that 
capture all of the support in a consistent manner. Even if adequate 
data can be found, substantial interpretation problems will remain, 
*\ Appropriations expje&sed on a,per -student basis must be examined 
within a context of understood limits and tempered expectations. 
Are these actually peer states in a meaningful sPnse, or cbmpetiton 
statest or just a set of stat^? lythe purpose of the comparison to 
a&sess relative costs? If so, some control must Ije established over dif- 
ferences in the services purchased by the appropriations. D*, Kent 
HaUtead is trying to control for some of the differen'ces^ith his con- 
cept of a "system cost index/* (see Magarrell 1982), If appropriations 
are expressed on a pef -capita basis, differences in state envifonmer^s 
also become important to legitimate interpretation of the meaning 
of state rankings, {McCoy and HalsteadI1982]go to great lengths to 
incorporate relevant filatures of the envi^nment in their study of 
state fin^ces.) 

Comparative studies among states that focus on concerns more 
specific than overall funding levels usually are less subject to com- 
parability problems. Lingenfelter (1982) lists some of these specific 
topics, "faculty salaries, degrees awarded, the availability of student 
assistance, participation rates, federal funding obtained, and profes- 
sional graduates imported or exported* *(p, 1852), He concludes thk 
such issues ha^e more utility than comparisons o£^ros5 funding 
levels* Howeyer, preliminary result of a recent survey by Hample 
tl983) indicate that a ntajonty of respondents consider Halstead*s 
.comparisons of gross funding levels to be useful Unquestionably, 
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those comparisons as well as the data on state appropriations com- 
piled by M. M. Chambers (and summarized in the Chronicle of 
Higher Education) receive a lot of attention. 

At the state level, then^ financial comparisons aresimilar Jo those 
at th^ institutional level and even more fraught with difficulties. 
Despite the difficulties^ such comparisons will contmue to be done, 
providing a needed frame of reference to lessen th^ impact of 
injurious provincialism. With higher education feeing one of their 
largest investments^ the states are not likely to ignore what might be 
learned from the experiences of others. 
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Developing 
Peer Groups 



When considering comparisons among onstitytiopSt the 
tenden^: V is to think immediate ly^ and pnly, oTcomparisDns 
among peers. Although various other kinds are possible* peer 
compatisons command most interest— and also put comparative 
data and comparative analyses to their severest tests. This chapter 
pto\^des, therefore* an extended discussion of how to go about 
dev eloping institutional peer groups^ By way of preface* we consider 
the ^sential nature Qf comparisons. 

In a general lexical sense* a comparison is an examination of two 
or more entities to determine similarities and differences. Diction- 
aries do not dwell on questions of inherent comparability a/nong the 
entities or compatibility of the data-in higher education, however* 
conflicting opinions abpfi^t the validity or propriety of particular 
comparisons arise because the comparability and compatibility 
issues are addressed from various viewpoints. 

In considering w hethet a comparison makes sepse, the crucial first 
concern should be the enveloping expectations or assumptions, 
sometimes specified but often implicit. Is it being assumed, for 
example, that the entities (departments, institutions, state systems^ 
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under comparison are essentiatty similar, and that what is being 
looked for are some minor differences? Or is it^ thought that the 
entities are dissimilar m irr!l3ortant ways but still worth comparing 
along some dimension or other? Misunderstandings in this regard 
can lead to confiasion in the development of peer groups and unne- 
cessary skepticism about the results of the com^rison* Fo^ illustra- 
tion^ consider a i^ase in which'faculty workloads are compared 
within a set of institutions that are peers with respect to several basic 
characteristics— enrollment t say* as well as research orientation and 
per-student expenditures* Against that, contrast a* comparison of 
fatjLilty workloads at institutions whose only common bond is that 
they fall within thesamepolitic^lboundary.Eithef comparison ma^ 
be vahd^ But s^eral kinds of trouble can be expected if the two 
comparisoiis proceed from the same set of assumptions. 

The point can also be illustrated by a reference to the frequently 
heard notion that it is improper to compare apples and orah^. 
Obviously^ apples and oranges can legitimately be compared in any. 
^^umberof ways, look into any book on nutrition* The admonition 
Is really a comment about invalid assumptions: do not compare 
them,on the assumption that they have no important differences— 
that an Ofange and an apple *are the same as two oranges or two 
apples* Similarly^ a workload comparison between community- 
college faculty and university faculty is questionable if one assumes 
that their workloads should be similar^ merely because they both are 
in higher education. Withqut such an assumption to cloud the 
comparison, something usefril may be learned from the differences 
in workload that are bound to show up* 



Assumptions 



In the development of institutional peer groups, then, consider 
first the assumptions that an analyst might reasonably make. AJcey 
analytic assumption acknowledges a pragmatic truth: no set of peer 
institutions is a set of pristinely identical institutions* To put it 
another way, an institution has peers in degree rather than in kind. 
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Institution!^ are never altogether the same> and never totally dif- 
ferent. Cuni^equentlyt those who want to develop a peer group must 
delineate JUfat how sinriilar the institutions must be to be similar 
enough. The degree of similarity required will be a function of the 
purpose^ use^ and focal issue or issues of the Lomparison—in shorty 
t[4 objeawei of the analysis. * 

The objectives of the comparati^ve analysis alsQ provide much of 
the context within which the analyst can decide m uhat ways the 
peer institutions must b&similar. The number of dimensions, or 
inatitutionaKharaLteristics, that might be included when assessing 
overall institutional sitnilarity is quite large. Because the relevance of 
J particular dimension depends on the objectives at hand, the 
analyst has a basis for paring down the large list^ thereby creating a 
data set that can be both meaningful and manageable. 

Those who m^ke comparisons cannot avofd deciding, either 
overtly or tacitly, the appropriate degree and manner, of similarity 
tha.t peer institutions should exhibit. At the most fundamental 
level, thereibr^e, human judgment plavs apart in thedeii^pmentof 
an> institutional peer group. This i/iescapable situation is another 
key element of prior awareness, for the analyst as well as everyone 
else involved in developing the comparison process. Recognizing the 
role of human judgment encourages the developers to begin by 
thinkings rather than by coaching fot a cafipe book. More funda^ 
mentally^ it erj^cQUragef the develppers t6 consider icfio should be 
inv olved in n^aking the judgments. If the developers could depend 
on bemg able to discover the whole tfuth and lay it at the doorstep 
4of the User, then partit:tpatioti lathe analysis would be a less impor- 
. tant i&sue> But that is not the reality with peef-group comparisons, 
and tho&e who igporethtflimitationsofsuch analyses should not be 
surprised if their results are ^ored. Finally^ recognition of thy role 
of human judgment will help prevent the peer group from being 
oversold, from being represented as more than a construct. 

If the ob>et.tive of the comparative analysis influeaces the way the 
peer group is structured, it follows that a change in objecttve may 
dictate a change in peer-group Composition, Over time, an institu- 
tion may put together several peer groups, each appropriate tp a 
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particular analytic objective. This is not necessarily a heavy cornpli- 
* uition, in and of itself. The real problem ties in the way the fruits of 
peer-group analysis are typkally used. How ujse would ft be to ^ 
: ^^efore a funding body ^ such as a legislativeooipmittee, with a pocket 
/ full of peer groups, one for eackissue (s^^ as firculty salaries or 

appropriations per student}? On its face, that sort of maneuv'er 
\^ould appear self-serving to anyone n^t well informed about the 
analytical process, - ^ ' - ^ 

Practkalty speaking, then, there is good reason to -incorporate 
muItipleVbjettives, to the extent possible, when developing an insti- 
tutional peer gtoup, A truly all-purpo&e group may be beyond reach. 
But a general-purfKyse group, useful on most occasions, may be 
feasible for most ir^stitutions, * ^ 



Variables 



We have mentioned the task ofMletermining which institutional 
charractetistics, or ditig^sions, are relevant to the kind of com - 
parison one wishes tfT/nake, ^ 

If a general-purpgee peer group is wanted, probably it is best to 
fo<-us heavily on dirnensions that describe institutional missipn and 
environment. Our experience suggests that most cornparisons ulti- 
matel\ are directed tothe assessment of resoufce dtiliiation patterns 
istjadent'facultyjatios, /acuity -staff C4(*Ds, expe^nditure^'per student^.;^ 
proportion of expenditure W^functi^Jit and so fOrth> If this is the ^ 
case, those patterns probalSty ought not to^gure in the selection of^ 
the peer institutions, Othenvis^j a sel^^^attng circnlarity will be 
built into the anal^is. In any case, the similarity of mission (ends) 
and environmeni^ ]rrK>ng p^r^roup^^nstitutiojis is what jnakes,^he 
comparison of their res<^rce^utilizati^ (means) interesting. I 

idedon,el<ch* 



ave been decii 



^'hen thedimensiortffor compari^<^ 
mus^«^p^ationa11y defined^ '^^^ ^^^^ mus^ be fdentified to 
constmTO the measure qrmeasJr^of^chdimensi 
fixation can be complex, Tbe dimensf&n &f '*si:e^"for exaifiple, could 
be represented b^opfof^Sjpnbinatioi\of variables, including total 
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hcad<ount enrollm^nts,^FTE enrollments, numbers of degree pro-- 
gramst doctoral degrees granted, total expenditures, and the like. It 
may not be obvious uhich measure^ or combination of measures, is 
bei^t* A dimension such as **quality of the academic program**can 
present another sort of problem. Of the measures , that come to 
mind^ at least a few, such as reputation and faculty pro^fucjivity, 
may generate controversy and perhaps weaken the credibility of the 
coTnparison process. (See Moden and Schrader 1982, and Smart, 
Etton^^ and Martin 1980, for exaT^lples of qualitative indices that 
have been used in a peer-grouping coritext J The problem of specify- 
ing measures (v ariables) for each dimension is compounded by the 
,*-nced to have access to all of the relevant data for all potential 
peer-group institutions. The data must also have been collected 
according to standard definitions and procedufS^so that they are 
compatible. {Practi^^H^pea king, standardisation is nearly always a 
matter of degree.) 

Relevance, acceptability, and availability are not the only major 
considerations in delineating the compaJrative-data set, A some- 
times equally important consideration is the ^inalytic procedure to 
be used on the data. X'arious options will be discussedlater. Hereit is 
enough to note that a list of a doien-or more variables almost surely 
will entail the use of multivariate statistical techniqueSj with their 
attendant.costs^ requirements for anal^ical capability, and process 
ramifications, ^^hat happfcns, for instance, to the confidence of a 
decisionmaker w ho has a large stalce in the comparative analysis but 
to whom the teclinique looks like a black box? 

\ariables*that operationaliie comparative dimensions are based 
on data that may generally be viewed as falling into one of three 
categories, depending on the kind of measu rement scale employed— 
nominalf ordinal, or interval* Nominal-level dat^have no magnitude 
and are discontinuous: they distinguish entities simply as being 
similar of diflferent with respect to specified characteristics* 
Examples of nominal variables include institutional control 
(public/private), religious affiliation (yes/no, or Catholic, Protes- 
tant, Jewish) and medjcal school on campus (yesVnoX 

* i 

47 




Chapter 3 



Ordinal-tevel data convey information about the relative order- 
ing of a set of objtcts on a continuum but nothing about their true 
magnitudes. For example, we might ask students to indicate 
'whether their parents* Income is less than $10,000 (category 1), 
between $10,000 and $20,000 (category 2), or greater than $20,000- 
^Latcgory 3)* With this information we would know that every 
' titudent in category 3 was from a family whose income was greater 
than the fainily income of any student in either category 1 or 
, category 2* For purposes of analysis, any two students in the same 
\ category are effectively treated as being from familie^*!^f similar 
income levels. 

Most institutional characteristics pertinent to.peer comparisons^ 
exhibit interval-stale properties. That is, their'^umeric values repre- 
*"SCTTt the true magnitude of the properties under consideration. The 
"interval" label refers to the fact that measurement units denote 
equal quantities along the continuum of interest. For example, the 
diffcrente between 2,000 students and 3,000 students is the same as 
the difference between 4,OC|5 students and 5,000 students. Interval- 
level data gre subject to auTarithmetic processes— addition, subtrac- 
tion, division, and multiplication— wid^out distortion of the data, 
Such information as revenues alJlP^expendityres, numbers of 
y students, and number of degree programs ar^^exaniSjespfsuch data, 
^ Given the characteristics of ordinal and intepVaTtevel data, the 
analyst can afways create an ordinaMeveTvariable out of interval- 
^ level data* Tfiis is done by imposir>gcut-off or threshold levels on the 
latter, so as to create? set of ordered categories, Incort^ data^ which 
IS inherently interval-level, is often treated in this manner— as we 
ourselves did in illustrating ordinal-level data, 

We 4re interested in this tripartite distinction— nominal, ordinal, 
and interval— because of the ways in which tliese different kinds of 
variables can Junction in various analytical processes. As an 
example, if having landgrant status, a nominal variable, is con- 
siderec^ssential for inclusion in an institutional comparison group, 
some 97 percent of the nation s colleges and universities are imme- 
diately eliminated from further consideration. On the other hand, if 
PTE enrollment, an interval variable, is deemed an essential com- 
parisortidimension^^ least one^dditional step will have to be taken. 
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Somehow » differences in cnrollmenc will have to becategonz(fd with 
respect to an inclusion rule. This may be done directly by estab- 
lishing threshold levels. It might be decided, for instance^ that any 
institution with not less than 6^000 or more than 9^^000 students 
could be considered/or the peer group. Alternatively, the interval 
variable might be used within a niuly|'ariate statistical routine, so 
that institutions with ^simitar enrotlrtients would be lumped 
together^ so to speak, without recourse to precise thresholds- 
^ In gist, we may say that' the proper selection of comparative 
dimensions in the peer -grouping process requires a prior knowledge 
of the objectives of the comparison, data availability, analytical 
capabilities, and process requireMnts. Absent this prior knowledge * 
and conceptual preparation, a hHJhazard and even disfunctional li^t 
of variables is liable to be drawn up, and another, iteration may be 
required. A set^ond iteSat^jftn is not necessarily bad, especially 4f it is 
intentional. Unplanned, it is detrimental to the credibility of the 
rhole uhdertaking. The old rule applies: take time to doit right— or 
[le to do it over, ' - * 




Analytical Procedures^ 



The c hoice of analytical procedure, like the selec^on of variables, 
should iake into account i^aFipus aspects of<he comparison as a 
whole* pbi^x)se, use, audience, available resources, and number of 
variables to be included in the-(inalysis. The analytical procedures 
discussed in this section include sectoring, Und cluster, factor, and 
discriminant analysis. All have been used, alone or in pttobination, 
Jtf develop institutional peer groups in 'higher education, wfe believe 
that most empirically .based.peer-grouping efforts incorporate some 
version of one or more of these four procedures. The latter thr^ are 
statistical routines that appeal more to the researcher than to the 
administrator. To a researcher, the main task in peer^group analysis 
is to divide the universe of institutions into groups of similar institu- 
tions. (See Terenzini, Hartmark, Lorang, and Shirley 1980i Elsass 
and Lingenfetter 1980,) The administrator, howevery^sks; "How 
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can 1 find a group of institutions that ar4^ similar to minerXheques- 
titjus are certainly related, and in fact^ the researcher*s approach will 
provide a pf^issiblesoliition to the administrative problem; one of the 
gr<iups iMjtated by che reMjarcher \m1I eontain the administraror*s 
in^itution. The questions are different, ngnetheless, and the 
analytical procedures they beget are usually different, too. We 
discuss the more research^nehted statistical procedures (in the 
plainest language at our command) because at some point, the 
administrator may have to deal With the results of studies employing 
them. Molreovqr, some admi rostra tors may find that one or another 
. of thCi>e procedures offers a viable alternative for addressing their 
particular coticerns. To our brief conj^eptual overview^ we have 
added appropriate references for those who want a more definitive 
and^technical understanding. 

{ - 

Sectoring ' - - ^ 

We speak of institutions as belonging to either the public or 
private sector. Of course, they can be assigned to numerous other 
kinijsof sectors, or categories, whose members arecharacterisedTfor 
instance, by the presence or absence of a medical school Qr an en^i' 
neering school on the main campus, or by having^(or not having) a 
religious affiliation or status-as a landgrant ins^ttution, or by being a 
single-sex institution or a traditionally black itistitution. In such 
cases, the sectoring procedure can ortiy be used with nominal 
' variables. Rarely, therefore, will sectoring alone produce a peer 
group. Thfe chance of that happening depends, of course, on the 
objectives of the ^fompanson— but also on whether the comparing" 
institution -happens to be usefully charaaerized by the nominal 

^ \ariable5at hand. ^ ^ 

The power and applicability of sectgringin peer^gjroup analysis 
Can be increased substantially by using it in conjunction with a 
threshold approach to ope or more interval va?rables. The notion ofi 
a threshold means that an institution is in or^out of a peer^oup, 

/^^depending on where it hes^n an interval scale. For instance,' a 
comparing institution may decide that only institutions withf 
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endowment earnings of more thar^^I00,0(30 atir^iJ^Ily tan be <:on- 
sidetcd fjcers. In this manner, one can add to the nominal vatiables 
mentioned above virtually any desired institutional (;hara(:teristic 
that can be specified in terms of interval variables. The peer group 
Itself can be described by fhe values attributed to each of the 
variables that delineate group membership, it might be defined as 
Lonsiifting of all institutions that meet the followi^i^nteria: public, 
landgrant, integrated medkal school^ 20,000 ormOre PTE students, 
$20 million or more in research eicpendity/esj and 30 or more 
doctorates awarded annually. Other criteria would yield a quite 
different peer group; private, Methodist affiliationj southeastern 
"United States, highly selective admissions policy, fewer than 1,S00 
PTE studenti>Y fodr year degrees accounting for, 90 percent or more 
of all degrees awarded, with less than 20*percefit in professional^ 

fields. , , ' , ^ V 

With the threshold modifi(:ation, se(:toring is a fastj efficicnnxrav . 
to develop a peer grouparound a given institution. The logic of its 
operation is readily understood by those who are not statisticians 
and is therefore relatively easy to employ in most decisioi^aking 
processes. In ideal clr(:umstances> perhaps, all interested parties 
would (:ome together to decide which variables to use and^ in tj^ 
case of interval variables, what the threshold levels should t^f- Bring- 
ing all the right people together may sometimes iDe a formidable 
undertaking. But when it is accomplished,,^ thc' tasks assigned to 
these people are straightforward^ and wh«t is. subsequently done to 
the variables they select is quite clear, 

^ From the perspective of institutional processes^ the sector- 
threshold approach does have a couple of potential weaknesses. 
One inheres in the very fact that it is so readily made a part of the 
decisionmaking process— which means rhat it also can be readHy 
polittused. The effect of each variable in the analysisy^illbe definite 
and quife apparenx, thus opening the selection process to game- 
playing of one kind or another Two, the inherent arbitrariness of 
the approach may become a point of attacfl for anyone unhappy 
abojiLxtie resulting peer groupOThreshold levels, in particular, are 
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vjjlnerable. After all, if institutjons ^vyh I.SQOgr fewer students are 
acceptable as p^^rs for a 5j:hool with 1^400 ^tudentSf how can one 
argue perisuasi\elv-t hat an institutjon with 1,5 10 students is thereby 
not a pecrMIn either case, the analyst may reasonably bcexpected to 
defi^r toth^; human-relations skills of the administrator in detef min- 
ing who shall dBMuct the defense.) 

We have observed that the administrator and the researcher 
usually have someuhat diferetAt interests when de\'eloping 
groups. A related issue should be mentioned, in co^unction with 
'tKesectonng'thrtJshold technique. It %o happens that this tec hni(ju^ 
IS the basis for most of the classification systems that have been 
developed for reporting on An^^ican higher education. The 
Carnegie Classification $ysteDn is best known. But the classifications 
cfevelopedby the National Center for Education Statistics ^J^fTES), 
NCHEMStand the American Associarion of Universitv* Professors 
lAAUn also arc widely used. Each classification is based on a hand- 
fUi of variables, such a^ control^ level of degrees awarded, breadth of 
Curriculum (th§ presence of a medical ^hool, in particular), amount 
of research, and number ofsrudents. And each system distinguishes 
among only a few typ^s of institutions, so that national trends can 
be monitored in manageable fashion, ^ " ' , 

From the standpoint of an individual /institution, groupings- 
suitable for national reporting may or, may noVtonstitute suitable/' 
peer groups**We have seen that the degree of li^titutional similarity 



required IS always a fUnctipn of the objectives of a particular con^ 
parative analysis. Generalh* a comparing institution will desirev 
more in&titutional sinjilarit^ than is exhibited ir^a typfcal national 
reporting category such fes "comprehensive univprstties,V It ought 
never to be assumed that a natjpnal reporting category will ade- 
quately delineate a set of peer institutions, especially when an insti- 
tution intends to compare its dat^^th that bf other individual 
institutions^ Comparing ones data with group norms for an appro- 
priate category of institutions wJthm a national reporting scheme 
would be less objectionahle, but still not best practice* 
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Cluster Analysis 



Thi;* a generar^me for a set of sratistiLal prot^edures designed to 
help identify groups of entities that have similar attributes. Cluster 
analysis is used where no a prion or theoretical information about 
the struLture of groups is available or assumed {SAS 1979). Cluster- 
jng proLedures are designed to work so that the entities in a given 
Llpster will be more like one another, wifh regard to the attributes or 
varmbles b^mg evaluated, than the entities in any other cluster 
There are various alternative clustering algorithms {see Hartigan 
1975). But in essence, clusters are formed by calculating the statis- 
tiLal distances between the entities and then grouping them on the 
bails of those distanLes, (U'hen this is done graphically, like entities » 
will appear in groups, or clusters,) The distances may be measured 
between values^ for institutional characteristics {for example, 
number of students or number of degree prograrps)* And they may 
be measured between combin^tions*of variables representing con- 
structs of those character is tkis-tttch as doctoral emphasis expressed 
as a Dprcewrage of gradu^e degrees awarded or applied-sciep<fe 
em^asjs similarly measur<kl. rnstitutions can be rank ordered on 
the basis of their relative distance from ^ target institution,' or the 
entire study sample ^ujiiverse) of institutions can be*divided into . 
clusters of relatively similar institutions. 

Cluster analysis overcomes at least one of the major problems 
associated with sectoring approaches. Clustering routines can be 
employed with continuous (intervaMevel) as well as discrete 
(nominal -level) data. Hence they d& away with, the ne^ for the 
-analyst t^> makf arbitrary judgments about the appropriate thresh- 
old levels or cut-off points for. interval variables included in the 
analysis, SuLhjudgmentSj we have pointed cnjt, are po^ibfe sources 
'of political controversy, ' 

But clustering approaches als^iave drawbacks, four of wKjch 
deserve mention here. Fiij&t, mcJRthej^adily available clustering 
programs L an handle^onl^a limiTfed number of cases and variables^ 
few will hajjdle more than 20 variables on even the largest com- 
puters. Second, as the number of variables increases, thepiobabUity 
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, increaifCi that essencjally the same dimension ^iH be redundantly' 
spciwificd. As a result, a particular dimension tnay be dispropor- 
tionatcly \\cighted in the analysis, for example, total enrollmenc 
1^" and total number of programs arc likely to be highly correlated; 
either one, therefore, could be caken as a measure of siie. If both are 
used in a set of five variables, say, then the dimension of si;e would 
aL^wOunt for^t least 40 percent of group variation— and perhaps even 
mure, depending on how total enrollment and total number of pro- 
grams vprrelate viith the other variables in the analysis. Of course, 
this is not a problem if it refl'ects analytic intenr 
. A t hird d ra wbac k is t hat most clustering proced ures , unless ot her- 
viise specified, give equal weight to all variables entering the 
analy&is. For complicated technical reasons, this requirement could, 
in (.ertain circumstances, create misjeading appearances of similarity 
or difference betvieen some instuuuons. The analyst can avoid this 
situation by assig;ning suitaBle weights to the variables. Unfortu- 
nately, doing so introduces an arbitrariness much Jike that 
encouhtered when establishing thresholds as part of aL^ectoring 
technique. 

Finally, cluster analysis does not tn itself provide definitive solu- 
tionsv Human judgment is still involved in deciding both how and 
^ \\here group boundaries will be drawn. Sometimes the clustering 
routine will reveal clear-cut, natural boundaries— but not often for 
the entire sample being studied. ^ 



Factor Analysis 




" Like cluster analysis, this term subsumes a fairly large number of 
procedures (Nie, Aull, Jenkins, Steinbrenner, and Bent i975). The 
most distinctive characteristic of factor analysis is its data-reduction 
capability. That is, it enables one to see whether some underlying 
partem of Relationships exists that will allow the data to be reduced 
to rnri^r set of f^ictors, as they ace callefl. Factor analysis typically 
IS used as a preliminary step to obviate some of th^ problems 
associated with overspecifjcation— in other words, to avoid having 
numerous variables that measure the same dimension. It also 
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accommodaies for the Iimifatron on the nutnber of variables that 
can be usedyfn a sector or cluster analysis/ Ideally, each factor 
measures a uifferenty^nique institutional charaLteristic, the 
fjLtor scores assigned to caL h instJtutiyp ordmanly will be far fejver 
in number than the original, ravt variables with whiLh the analysis 
begin^ Factor analysis usualfy is used in conjunction with other 
techniques and generally is nSt used to generate grtrnps of similftr 
entitles directly. One could in fact dp so, however, by using what is 
called a Q-factor analysis (see Stephenson 1953), The more common 
approach is to employ a sectoringorMuster analysis tg generate peer ^ 
groups from th^ factor scores. For example, total number of 
students, total number of degrees, and total expenditures for educa- 
tional □ndgenera^ purposes ar^ descriptors whose values are likely to 
be highly correlated. If thii turnsout tobe thecase, the^ descriptors 
will gejierate a factor, which the statisrical technique isolates sfnd 
which the analyst subsequently would doubt interpret as "sue," 
Each institution in the analysis is given a score that locates it on the 
si;e dimension. This score replaces the three ongitral descriptors^, 
thus reducing'the data set employed ia further analysis! y 

Numerous computer programs i^re available for performing factfr 
analysis. But the cookbook statistician should be aware oftmploy'c 
ing this technique. Proper use of factor analysis involves recognition 
of the assumptions in^tied afhd problems inheren^n the factor- 
analysis model, Irt^some^ircumstances, they irAy constitute serious 
limitations on the value of the technique, (Comrey -1973 and 
Harman 1967 are among the many basic texts providipg excellent 
discussions of factor analysrs and its furdamental assumppons,)^ 

We would be amiss ndt to mention a few common problems with 
factor analysis, ^ven at the risk of becoming too technical. First, as 
with most statistical techniques, the reliability of results from factor 
analysis is directly related to sample size, Comrey (1973) believ^ 
that analyses based on samples offewer than 300 cases have only fair 
reliability, Second, an tmderlying assumption of factor analysis is 
that the variables employed have normal distributions. Our experi- 
ences suggest that with respect-t^ colleges and universities, many of 
the Variables of interest will be seriously skewed,^ truncated, or 
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bimodaL Third, we have no precise machematical way to det^q|^nc 
hnw^many factors should be extracted. Thus one can never be 
Lcrtarn whether too many, or too few, have been extracted. An - 
inappropriate choice may disrorf the factor^solution. 

A sitatistn^al procedure called spatial-cortfiguration analysis com- 
bmcs factor analysis and mulcidimeneional scaling to form peer 

* groups. The technique has been usedprirnaril^by researchers and is 
not included in most statistical so^ware packages. So we will not say ^ 
more about this procedure here. The interested reader will find good^ ' 
iJiscussions in Cole^and Cole (1970) and Smartet a[. (1980). 

Discriminant Analysts ^ 

* This statistical procedure ts not used directly to form peer grouf>s 
tn the first instance. Its most frequent application has been afrer 
(.luster analysis was used to identify institutions most like (or unlike) 
one another. Clustering , algorithms seldom provide complete, 
definitive group, structures. In many cases, indeed, the analyst 
ultimately deader where group boundaries are to be drawn. Having 
thus assigned institutions to groups, the»analyst may then employ 
dtscriminant analysts to assess b©th the "goodness o^ fit*' between 
institutions and their assigned groups, and the relative statistical - 
contribution or influertce of variables employed in the cluster 
analysis. Uke factor analysis, discriminant analysis may be per- 
formed \^ith a- number of computer programs n^ available. We 
caution, however, that use of the techijique by those not well versed 
in statistics may lead to interpretation ^lificulties, (The interested 
reader will find discussions of discriminant analysts in a number of 
texts on multivariate statistics, including Tatsuoka 197r,Ftnn 1974t 
and Bock 197S.)' \ ^ ' ' 



_^ Illustration 

Much of the preceding discussion has been abstract So at this 
pointy the reader will perhaps appreciate a fairly comprehensive^ if 
hypothetiCjal, illustratipn of how an institutional comparison group 
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might be developed. The prcM:edure we have chosen to illustrate is 
often used at NCHEMS when we respond to requests for assistance * 
in finding institutions appropriate for comparisons. The procedure 
js not particularly sophisticated; it is relatively easy and inexpensive 
to implement. And it will allow us to demonstrate the application of 
some of the principles of de\?elopment we have discussed. ' 

At Oiu: hypothetical university, the rationale for seeking com- 
parativedata isto^eta widerperspectiveoa whether the proportion 
of.total expenditures allocated to instruction is reasonable* That 
proportion has been gradually declining for several years, a fact that 
is causing growing concern among the faculty. There is consensus 
chat it would be usefol to knov what has been happening at similar 
institutions. Our institution has rwt made extensive use of com- 
parative data in the past, $nd no list of similar institutions, accept- 
able to all concerned parties, ha§ been compiled. General agreement 
exists that some empirical basis shoyld be used.to determine which 
institutions belong in a comparison group. 

The first task is to decide what institutional characteristics, 
\ ariables, are to be used to establish similarity^ Several questions are 
involved, bec'Sause the issue has to be addressed in both general and 
specific tcrm^. The general problem is to determine a basis fof select- 
ing variables. The two major optionsusually are to ijse either institu- 
tional mission or resource-utilization patterns, or bothn The former 
h most often appropriate, because resource-utilijation patterns are 
so often what is to be evaluate^!. Indeed, this is the case with our 
hypothetical institution. One could start by finding a set of institu- 
tions with utilijation patterns similar to those a( the comparing 
institution^ But this would indicate little if anything about the 
reasonableness of the patterns#Dataon the respective institutiojial 
mistetShs ^till would be needed^ to give comparative meaningto their 
utiliJation patterns. We submit, then, that in most instances, one 
ought to start by stipulating that at the minimum, institutional 
similarity will mean similarity of mission, 

What specific variables can be used to characterije institutional 
mission^ or "role and scope*'? Table; 1 displays the modest list of 
variables with which we begin at NCHEMS when the comparing 
institution is a four-year college or university. Group A criteria 
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. TAbLE 1 
' Criteria for Comparison Institutions 
For 



A. Selection t. 



: / 





Iitftltutfon 


- 


Very 


Not 
Jmportant 


Control (PobUc /Private) . 


PubtriS 


X ■ 






No 


X 




X1(,\J)cal School 


No 


X 




Urban /^ural 


Rural ^ 


X 


1 






1 * 





B. Ranking 





' Your 
Kiutitution 


Range 


Check one 


ImpoTUni 


Important 


Not 
Importiot 


TotalJTE Er^rollittcnt 


8,055 


10,000 


„ X 






%AA Degrees ' 


8.1 


0-1 s 




X 




% BA Degrees 


77.5 


60-90 


- X 


It! . 




% MA Degrees 


H.4 


9-20 . 


X 






% Ph.D. Degrees^ 


'•0 


0-1 


X 






Prafessiorial I>egrees 


0 


\ 0-0 


X 






% Degrees ii^ IVofcwional 
Fields 


77.0 






X 




% Research Expenditure^/ 
In scr uc ti on Expend i tures 


•.002 






X 


• 


% Part -time Heiidcount 


17.4 


8-30 
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iniwtudc nominal v£jmbles that idenciPy an institution in clear -ci4t 
hishion: for instance an institution cither has or does not have i 
medial ^(whoul as part of iTjj^opetation— a fact that has great conse- 
qucnLCs for resource utili:ation> The list of identifying variables 
Luuld include such features as religious affiliation (either ycs^nu^ or 
in terms of a specific affiliation), Mngle-sex enrollment, and predomi- 
nantiv bl*:k enrollment. Group A includes an urban-rural variable^ 
that may be a proxy for some aspects of institutional mi^ion. It also 
rnay reflect certain environmental pressures on an institution that 
c<^ld affect the way resources are utili:ed» In a similar veir^, regional 
location might be important^ or the selection might b^ limited tjb 
inatKutions within specified states. 

Group B variables in table I are of the interval type; they must be 
assigned ranges to function in the analysis^ Again, one might want 
to add variables* such as the percent of minority-group students, or 
an index of admissions selecUvity^ or a particular program emphasis 
(especially if the comparing institution can be re^djty identified in 
that fashion)* Group B includes the particuUr variables shown 
partly because they have clear implications fou resource utilization. 
They also are representative of^the issues thtt consistently have 
come forward m our^^wn research efforts at NCHEMS to describe 
institutionaf mission by analyzing 100 and more variables* usingf 
factor analysis* , \ * 

^ e have said that tnc^riables shown in table 1 are appropriate 
for" four -year institutions. Another set is required for two^ear 
coll!|ges. However, some overlap will occur, *among both the 
nominal and interval variables* with those shown in table K The 
most significant differences have to do with program and degree-* 
level descriptors* 

The next step is to use the variables to generate a list of possible 
comparison institutions* In the NCHEMS procedure, Group A 
variables are used as idtQlxm criteria, Bor each nomin&L variable 
checked"inriportant"by thecompartng institution, a yes-nP decision 
rule i& inserted in the computer program used to facilitate selection. 
Our hypothetical comparing institution is publicly controlled and 
deems that characteristic important inselectingpeers. Therefore^ all 
private institutions ace eliminated from consideration forthwith* 
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The nommal^ variables thus typically pravide a quick way to pare 
du\vn the ljst*of possible compari&on instttutrons. This point may 
^uum tri\ latly obvtous. At NCHEMS, it ts tn fac| important, because 
our data base includes alf of the more than 3,300 colleges and 
universities included in the HEGI^suryeys. The elimination of 
some institutions from further vons™ratton at thel^utset h^lps in 
making the project fnore manageable? ' . * 

proup B variables are used tqxank -order the re5raim?1^ institu- 
tions in terms bf chetr relative "closeness"* to the clmpapng insti- 
tution. A 4-andida.te institution will either land with^or miss each 
of the ranges estaWished by the comparing institutipn. The candi- 

. date institutions are asSigned points for each mist^ and a p^nt total 
for each institution is calculated. In addition to this simple sum, a 
\\eighted sum is calculated, using the importance scale. ^ miss 
counts one-half point if the variable is only **irrmortant" to the com- 
paring institution, rather than "ver^mportant,\and po points are 
added for a miss on ai\ unimportant variable. This weighted sum is 
then used to rank-order the candidate instttutiohs. Table 2 shows 
the rankings for 33 of t^ife instftutions that welte'ipost similar to our 

' hvpotheticat comparing^ or target institution, follpwipg thje criteria 
set forth in table L " ^ ■ ' 

Table 2 sho.ws that six of the candidate institutiop^ have the requi- 
site nine characteristics, all falling within the corresponding ranges 
of the target institution. Institution 7 misses the ran^e for percent of^ 
part-time students, and institution 8 has too low^ a percent for ^degrees 
and professional fields. Neither characteristic is considered "very 
important*' by the target institution, so the weighted sum assigned 
to institutions 7 and 8 is jess than the simple sum. In'the next group, 
9 through 20, each institution misses on one very important charac- 
teristic, and soon, Inanalyiingsuch a list, the comparing institution 
likely would elimmatt some of the institution^shdwn. For instance, 
institution 10 awarded 2 percent of its degrees at thedoctpral level, ^ 
and institution 24 had a rather high ratio' of separately budgeted 

tarch to instruction expenditures, t*erhaps both could be' 
mated, since these figures missed the specifi^;^ ranges by wide 
inarginsx The extent of a miss normally would brought into 
consideration at this point— manually, so td 5peak, since it^is n6t 
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mcludeJ jn the automated, (:omputer'based mat(:hing routine. The 
j^umpanng institution might also no^^- decide to (:hange the range 
limits oi; ont or more variable;?, rhctcby rearranging the rankings, 
or to add variables, which also' would reconfigure the group of 
poiisiblc Lompari&on institutions generated by the computer. 

ad vise those who engage in the ;>ort of sectoring -plus-threshold 
approai,h illustrated in tables I and 2 to work toward a final compar- 
-ison |roup of 15 to 2&j^nstitutions, A group of that sue is small 
enough to make the gathering of adJitiopial data on the member 
institutions a task of reasonable scope, should more data be needed. 
It is also large enough to dampen the effects of bad data, if such data 
are prc;>ent in subsequent analyses* Suppose a peer group of 20 insti^ 
tutu^nsb Uit%J in addressing the problem we setforouc hypothetical 
liistitution, the target institution in table 2: h an appropriate pro- 
pi^irtion of r«jsouri,e& going to, instruction? OfTeTjf the institutions 
might happen to be at the mean proportion for the group» let & say 
.45. But it might erroneously report a figure of ,54 instead {a 20^ 
perj^ent error)* The net effect would be to change the mean^ropor- 
tion for the group from a correct figure of »45 to an incorrect *4545» 
The erroneous mean proportion, which is^the normative infor- 
mation sought by the comparing institution^ is only in error by 1 
percent— not enough to seriously mislead anyone under most 
circumstances* 

Finally, we encourage institutions benr on comparisons to use the 
team approa\.h ipi developing both the initial criteria and the final 
peer group. Adminisrrators and faculty on any campus can provide 
a lot of information about other institutions. Why not make use of 
that information to help ensure the validity of the overall process? 
Clearly t the relatively few data elements employed in the process we 
have illustrateJ leave much unsaid* Knowledgeable people can often 
fid in the gaps. And, as we have previously stressed, it*s just gcrad 
strategy tu involve the concerned parties from the ourset rather than 
prc^senring them wirh a fait accompli, hoping that they will accept 
the judgments and ^compromises imbedded therein* For anexamjplj^ 
of a successful teatn approach involving pjjrential adversaries, see 
Teeter eraK(1982X * ' ^ / ^ 
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TABLE 2 

Possible ComparispfTlnstitutions for Target University 
Public> Non^Landgrant, No Medical School* Rural 
Very Inipt = %BA, %MA, %DR, %FP, Res; Instr, TotFTE 
Impt = %AA, Res; Instr, %PTHC* %Degs Prof Fields 
Iteration ^1 
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22.1 
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■ .0 


.0 


6162 


' 89.9 


10.1 


0.0 


0.0 


0.0 


.73.5 


.019 


21.2 
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.0 


.0 


8210 


87.4 


, 10.3 


0.0 


0.0 


• 2'.3 


66.1 


.021 


15.8 , 
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.5 


1.0 


6699 


74.2 


15,7 


0.0 


0.0 


10. 1 


' 69.1 


.010 


32.0 ^ 


8 


.5 


„1.0 


9789 


84.5 


• 15.5 


^0.0 


0.0 


0.0 


'■55.9 • 


.036 


9.1 
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1.0 
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8104 


82.7 
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0.0 


71.3 


.041 
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.090 
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In Conclusion 

' ' 1 ' 7- 

Employing complex statistical procedures such as factor and 
Llui^tLT analysis is certainly a legitimate approach to Jcvelopir^g peer . 
groups. Indeed, some writers Ajch as Tereniiniet ah (1980) make a 
strong Last for their use m preference to the.simpler merhodology^ 
illustrated above. Several cautions are in order, however The 
scacisticarl procedures will probably look like black boxes to most 
people* Despife that appearance, the procedures stil! depend heavily 
on human judgment. And while tbpy are easy to emplny, they are 
difficult to thoroughly understand. In sort, the heavy statistical 
approaches are vulnerable to challenge in the po|^itical atr of the 
decisionmaking arena* and often in the technjcal arena as weltv^ 

By and large, we BSslieve that an institution seeking to develop a 
group of peer institutJons will be Better off employing relitiveiy 
simple analytical procedures, while spending most time and effort 
on the overall process— the objectivesj political issuesj communica^ 
tion, involvement, and so foVth— within which data on comparison 
institutions are to be developed and used. 
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State of the Art 



In this chapter, we con&ider the current situation with respect to 
comparative data from several vantage points: the major sources 
of data, what we know of the qu^lH/ of at least some of th^se data, 
and the stat^ of the analytic art. We conctude with some administra- 
tPve guidelines Regarding data quality and data presentation. 



Sources of Comparative Data 



An institution may uhtain data on other collies and universities 
directly from those institutions. We willcJiscuss^thar approach later 
on. The bulk of t he chapter concerns secon<3ary sources, from which 
most comparativedataonins^tutiortSofJiigher education emanate. 



Federal Sources 



^ — / 



The basiCj broad responsibility for alllevebof education belongs 
to rhe. states. Among rhe several ipecialned roles taken up by the 
federal government is the gathering, o(Af|tionwide statistics on 
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(woilegei and unrversities. Data on a few higher -edu^:ation variables 
go bauk far as the Census of 1870* Th^ U*S. Office of Education 
first i^ent surveys to institutions in 1929-30, The-(:ijrrent, most 
broadly bailed federal effort, the Higher EdLn:atton General Infor- 
mation Surveys (HEGIS) of the National Center for Education 
StatistKs (NCES), date from l%6. The NatioOal Science Founda- 
tion (NSF) -surveys, another major source^ofjl natation -specific 
data, began in 1954. 

From the federal perspective, the main^purpose of HEGIS is to 
provide information for federal policymaking. The utility of HEGIS 
. data for interstate or intennstitutional comparisons is only a ^ide 
benefit from the federal standpoint, ^'hile being of primary imp^p ' 
tancc to many ^ates and institutions. 

The HECIS system at NCES consists of a series of surveys^ some 
LonduLted annually, others every.two or three years— or even less 
frequently. Oler the years, some changes have been made in both 
the iLhevJuIingand content of the surveys* Although completion of 
the HEGIS surveys is voluntary, compliance coDtinues to*run at 
about 90 percent— and higher than that if one ignores certain types^ 
of speciaKy institutions included in the HEGIS universe of about 
3,300 accredited institutiOHS^ As of 1982^3, HEGIS included the 
following surveys: 

Finances 

Enrollments ^ 
Earned Degrees * ^ ^ 
* Employee Compensatio J- v 
Libraries 

Resid<^cy and Migration ^ . . 
Facilities 

State Expcndj^u^es ' ' ^ * 

The First four surveys normally are done annually^he last, on state 
expenditures for higher education, was conducted first in the fall of 
1982. The only HEGIS survey that does not cot(tam institution- 
speLifiL data, it wjK have value for direct comparative purposes only 
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thestai;.e el but will also be an additional source for contextual, 
environmental in;lterial relevant to institutional comparisons. 

As the bur\ev tittat ^ndn^ate, the HEGIS system covers a wide 
range of topics thartfcHn^bc useful ^% comparative purposes. The 
>pccifR contents of trnr tndicidual sur\ eys cannot be reviewed here; 
.idministrators should have no troubteMocatmg copies, however. 
Suffnw(*it to say that muiwhof tlie basic data one might need for ir^ti-^ 
tutional (wOmparisons are included in thfe HEGIS system. It is not ' 
complete, of course. Examples of data elements not in the system 
include the number*of part -time faculty, the number ofsupport staff, 
student (wfedit hours, noncredit enrollment, some reserve funds, and 
the amount of student aid provideS frpm institutional sources. 
Neither ^wost^ nor enrollments a^'t^vailable at the departmental or 
disapline level. Thus (wOmparisons based on HEGIS must essentially 
be institutionwide. 

The J-[EGIS data base is truly anenormous resource, and ijosmall 
uchic\em^?nt. The data are in the public domain, apart from sorc\^ 
minor exceptions b^iving w do wrth some faculty s^ilanes. Thed^ta 
base can*b^ accessed in t^&i^^tially three ways. Data can be pur 
chased in maiwhine-readaWe fozm on magn^ic tapes dir?cjtly ftbm 
NCES, This sh^ld4:)?^j0ji?cjH 

staff and computer equipfCfent^K*i wiiSngcnrsslohave staff spend 
>oinc time be^op'ing f^t!rar^^rt^tKc>^ipc Special reports ^ 

can be reflu«ted from KCB^^anJ" the agency als6 publishes annual 
reports based otiTiEGiS^^XR&^d^ta^ia the p^iblished Reports {for 
GxaSnple, Condition o| EJucoLop^and Digest of Edw^nm Statistics\a\^ 
highlv^ aggregated, tut they ao providesome instittttiohal nortns for 
Lom5ar3tt\ejmirp0&e5. FtnalW, ^Qal^^ticafl^ miftd^d organiza!tons 
that regularly ^ork ^ith the MEC3&daia^a^^^ he a source for 
-4dEPIS daia. ^'^ave alre^idy put forwara*fhe best example. Since 
. 198&^ fcirnrtal procedures ha veB^en m place atNCHEMS topfowde^ 
administrators an^analysf^ipDuri^rh? country with access to 
\irtualty the efitire ^lECl^ 4**^*-i^?5*lC^?^^'^ 4'^^^^ frorn the 1960s 
may ndt be availably f^." r,^^* 

The surveys of ^hf.TlatS^nalrSciehc^ Foundation are another 
largpiscale, but somewKat Ie«. ^mb^ious, federal data-collection 
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, effort, .NSF does these four surveys related tb higher education 
annual 



Graduate Enrollment in Science 
Science ahd Engineering Personnel 
RStD Revenues and Expenditures 
Federal Grants and Contracts 



The first thrjee are based on institutional records, the last on-che 
tecordsof fedetal^g^ncies. The universe of institutions is far smaller 
than the HEGIS univetse'and varies considerably by sur^y. The 
content js^lso more limitedandfocused, and yarite year to year. But 
It does permit certain financial comparisons at a lower level of 
aggregation— fot instance, at the level of "electrical engineering"— 
than does the HEGIS syste^n, NSF publishes a number of diff^ent 
reports based on these surveys, some of which contain institution* 
specific data, JjISF also sends annual reports containing nornjative 
data to participating institutions, and through a subcontractor, it 
sells magnetic tapes containingthe survey data. Alternatively, NSF 
data can be obtained through third parties such as NCHEMS. 
While current HEGIS tapes contain one file per survey per year, the 
NSF data are maintained and provided as lof^itudinaf files. For 
example, onp tape will contain 10 years of data on graduate enroll; 
ments in the scieiices. {Longitudinal HEGIS tapes are currently 
being developed,) ' 

The Office of Civil Rights also conducts surveys of colleges and 
univ,?rsities, The/resultirig data could be useful foi; a4imked'setxtf 
-comparativf analyses. HoweVer, they are*not readily accessible. * ^ 
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The trefnerjdous growth in public higher edUc^tion after World . 
War U \tas a strong stimulus to statewide coordination and planning. 
Most plannijig efforts included an information-system component. 
Man> states now maintain extenstvedatabase^on their public it)sti- 
tutions, and sometimei data orv private institutions 3f well In some 
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instances, the stat^ata base (pomades with HEGIS data for state 
insntutions. In other cases, state data may £»e more or less detailed 
and mjv include data elements different from those used in the 
■ Hingis system. * . ^ ' * ^ 

Differences among the states in population^ the role and scope of 
theiL higher -education institutions^ reporting requirements, and 
iuch make generalizations (difficult- But fri pursuing state-level data^ 
contacting coordinating agenUes or institution-level boards of 
regents or trustees usually worthwhile, A quick way to get a sense 
of what is avaijable in published form from all of fhe states is to work 
through the State*l-Jtgher Education Executive O^icers (SHEEO)< 
Their office and library are currently housed at NCHEMS< 
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The natioui's three regional higher -educaxion associations are also 
sources 'of comparative data. They include the Southefn'Regiopal 
-Education Board {SRE3)^ the New England Boardof Higher Educa- 
tion (NEBHE), and the ^ ester nJnterstate Commission for Higher 
Education (^tTCHEX These 'organizatio^ are especially useful for 
interstate comparisons in theirjrespective regions but generally less 
so for interinsritution^l comp^isoijr. ^ ' 

Nurnerous special-interest associations gather and maintain data 
on indiv idual institutions .or can at least provide ndrmative data by 
type of institution. These sources include at least the following {and 
there may be others): ^ " , - * . 

A^merican Association of Colleges of l^ursing ^ * 

^American Association of Communitj? and'Junior Colleges 

American Association of Medical Colleges ^ ^ 
. American Assocratioii of Stare Colleges and Unlvejrsitries 
American Association of Universities 
j^merican Association of University Professors 
Association of Physical Plant Administrators 
Associarion of Resegjx:h Libraries 
College; and University Personnel Associarion 
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'College and University Systems Exchange ^ ^ 

CounciJ for FinaYlcial Aid to Education , 
Council of Graduate SchoHols 

National Assoctatton of Independent Colleges and Universities 
National Association of State Universities 3nd Land-Grant ^ 
Colleges 

Natibhah Association of Trade *and Technical Schools 

A(f(:ess to the data maintained l^y these associations, both regional 
and spe^:i^il interest! '^an be obtained through their^ublications or 
through direct request. 

Other Secondary Sources A 

Several universities gatherand make a\ailable particular kinds of 
comparative data, includiiMphe UnV^rsity^of Alabama (statistics 
on schools of education)^ Oklahoma State Uniyersify (faculty 
salaries for a sample ^>f state colleges and universities^ ^and the 
University of Arkansas ^adipinistrative' €alaries at^dc^tora'te- ^ 
granting universities). The University of* Arizona has developed 
whot is probably the cleanest longitudina We ta file oh enrollment 
Row^available— for atl accredited institutions, dating frqm 1965- 
Again, ^;Jiese datajare acce^ible in published reports or by cont^ct- 
ir^ the institutions,' 

Varjous special studies are published that^occ^tonaHy contain 
normative data useful for comparison purposes, Perhapjs the best 
known i5 Alexander Astins annual^ Coopei^ve Institutional 
Research Prograffi'(CIRP) study of the background, attitudes^ and 
goals of Loll^ege freshmen, M, M- Chambers does an annual survey 
of state Appropriations by state and by institution. Portions ^of the 
Astin and Ghambeirs surveys appear regularly in the Ckromde of 
E*JuaiCii3n. The Ckromde also carries a series of otjier surveys 
on a rapge bf topics^ these can help esi;ablish norms^sftype^of insti- 
tution. The American CounciLon Education (AOfc) us^s a^'panel of [ 
institutions for surveys on ^ues of current interest; i^Its are 
available in ACE publications- The National Association of 
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Colleges and Univ^ersity Business Officers (NACUBO), in conjunc- 
tion with the American Association of^ Community and Junior 

Colleges (AACJC), doe^ an annual finantial survey of a*sample of 
community colleges; the results are published in NA(^UBO*s 
Business Officer, john Minter and Houard Bowen have J&ed samples 
of both private and public colleges an^i universities jto do annual 
assessments of institutional financial well-being. These dat« m%y be 
useful as normative for some types of Institutions. Portk>T;\s appear in 
the Chronide of Higher Educatiqnt and the full studies are available as 
separate oublications. The National A&5CM:iation of Stete Scholar- 
ship and Grant Programs publishes the results oTitsannuatSurvey of ^ 

^ srate'student aid; these data can be used only for state-level com- 
parisons. Higfier Ediicatim Financing in the 50 Stai^i^ by McCoy and 
Halstead {1982), is a 50D-page document containing an extensive set 
of comparative data on the states and thefr institutipns of higher 
education,^ botK public and private..NCHEMS ptans*to publish 
updates of this study annually- * 

The largest collecrion of liierature on higher.education is found in 
the Educational Resourcesi Information Cenrer {ERIC)* System. 

,AIrhoii^h jt is nor a major^source of comparative dat^, the ERfc col- 
tec tipn contains numerous one-tune studies, some of which contam 
data potentially useful for comparative purposes, Ihde^d* it is a 
primary source for daf a about such topics as attrition and retention, 
whtbre nd national surVey data are available* The collection can b^e 

.searched electronically through key-word ^ystem/ for which 

'indexes ar^ available in most corege libraries- Many university 
librahes maintain the entire 'ERIC cotjecnoi^, which currendy ^ 
numbers roughly a quarter-million item^ti in microfiche form. \^ 

E\ery yeiar, a set of publications appears that provide sn^pSsi\bt?,of 
most of the nation's colleges and universities. The College 

A^ovejoy's Guide to Coi/ege^ the College Mandboolf, B^imn s OwSIe to 
Colleges and Universities, and Cas^ and Birnbaum s Ouide are well 
known. These publications afe ^specially useful for institutiort- 
specific data on institutional selectivity— test scores, class rank, 
applkation-acceptarice^ratios, and such* these data ^re not readily 
ayatlable el^where- Tne institutfonal snapshots are aisp handy at 
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the stage when a comparison group is being developed. The College 
B(.>ard publishes more specialised documents^ the College Cost Book 
and the Index o/Mujuri, which are useful for comparing tuition rates 
<ind curricula across institutjons. The annual Higher Education 
Dir<;ctor> contains just a few data elements.relevant to comparative 
anal>;ses, but it covers virtually all accredited instjpntions and 
indicates both institutionwide and program-spetific accreditation. 

Institutional Arrangements 

Occastonaltyt colleges and universities will cooperate^ i^r^uie 
and share comparative data. The immediate goat may be a o ne-rime ' 
study- A ^welt-known example is the California and WesteraCdn- 
ference Study (Mjddtebrook 1955), acost and productivity analysis ^ 
involving a dozen research universities. Soihetimes^ dafa-exchang^ 
procectures will be established as part of a long-standing formal 
association of institutions. A good examplp is the' American Asso- 
ciation of Universities^ wltich includes among its activities a limited 
sharing of data. Institutions with common sponsorships may also be 
'involved in data'exchanges. For instance^ the MINDS data-sharing^ 
systfem serves more than 70 privatejnstitutions affiliated with the 
^Methodist church. These formaUrrangemerits usuMl^involveinsti- 
-tutions with generally similar missions,, and thus*provide a good ^ 
start if that sort of peer Comparison group is desired. Innumerable 
♦informal *^data -exchange arrangements must exist across the 
country- They may be paft of more comprehensive cooperative 
efforts, or established entirely for their own s^e. Sometimes, for 
example, institutioo§ m a particular iirban area that have littie more 
in common than location will regularly share data. 

^ [ : ' 

* Dat:a Quality ' * 

B^ause comparative data are available from a multitude of 
sources, the overall quality of such data is diffic ult* if not imposstblej 
to assess. Furthermore^ the concept of data quality is itself complex, 
and certainty open to various interpretarions- Quality might be 
^ ; — 
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taken CO refer primarily to (:ertain inherent properties of the data, 
hULh aCTijraLV^'BiJt it might instead be taken to refer to pSinicular 
Lifecs of the dat^., such as strategic planning versus efficiency com- 
parisoni., or to the majitleTrEri^hiLh the data are used— for example, 
m dii.aggreg.ited rarKer than aggregated form. All these issues have 
Something jo do vCith cjijality in a broad sense, ^ 

In light of the mulnfaceted nature ofdaca quality, we will suggest a 
fev^ general pnnuples that the administrator can employ in making 
a prion assessments of the quality of datain particular situations^ We 
will also took in more^ specific ^terms at HEGIS, because it is the 
major source of comparative data accessible to all* By way of intro- 
ductjont we wilt briefly discuss inherent dataproperttesthat have to 
dt^vvuh quality* 

The quality of data h#*e<hree tnher^ent determinants— validity, 
accura^t and reliability. Each property relates to measurement. 
Validity concerns the extent to whic h the data actually measure or 
Lode vvhat they are intended to describe* Assuming for the momeqr 
that SAT Scores are goo4 indicators of academic ability, ^is the 
average SAT score of entering freshmen a valid measure of the 
aca3emic ability of the enure undergraduate student l^ody? In the 
context of comparisons, validity takes on additional meaning. Firsts 
to extend the SAT ejfample, note that the same measure— average 
SAT score— cuuld^ave varying degrees of validity from one institu- 
tujn to the ne^t; contrast the institution where most entering fresh- 
men take the $AT exam with one where Qnly 10 or 15 percent do* 
Or contrast the highly selective institution, where 80 or 90 percent 
of enferUig freshmen are still around as seniors^ With the less selec- 
tive institution that loses 30 or 40 percent of its freshmen after the 
first y^3iv. Second, \^hLle "SAT score** has a denotation that is likely 
to be consistent from one place and time to another^ lots of other 
measures do. not have a similarly consistent denotative meaning; 
number of FTE students, cost of instruction, amount of student aid, 
number of programs, revenues from gifts and grants^ and soon* Th^ 
list i^ong, and it contains many descriptors ofgrerft interest from a 
comparative perspective. In shorty when'doing compariions^ t?ie 
validity of data becomes more difficult to assess* The measure and 
the concept should be well^atched* Al^o^ the measure must be 
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equally appropriate in other locations and must be performed in 
similar fashion in each of those locations- ' ' 

hc\Mro^ and it\\ah\\\tj also become more problematic in a com- 
parative mode, but neither concept is ambiguous. The exrent to 
\\*hich measurement \jr coding is free of error (accurate) and will 
agree \* it h the measurement or codmg of another observer (rehabte^ 
bev-omes more problematic because comparisons require that dat^ 
be recorded in more than one location or circumstance. The 
building of a national^ (Comparative data base is the*^treme case. 
Imagine, if you wjl, the situation each fall as Jiterally /thousands of 
registrars count {possible error) the number of various types of 
studetits {possible error) and encode the results on a survey form 
^possible error)! The forms are sent to a central location and^heir ' 
contents are entered into a national data base (possible error). 
Reports are generated from that'data base (possible errof). Data 
from those reports are then incorporated intodocumentsfor use at a 
particular mstitution; the incorporation may involve merely copy- 
ing the data^ but even so simple a procedui^ harbors potLemial^jfr 
error^that can diminish accuracy or reliability. 

We think that any empirical comparison {tnat is, any comparison 
based on Qata) will involve data that are to some degreejnvalid, 
inaccurate, or unreliable. In chapter 2 [p. 21], we^ suggested ways to 
minimize fh^ threat of data error in financial comparisons* Here we 
want to address a related, though soiji^what different, concern; 
How can an administrator quickly assess the likelihood of error and 
the consequences thereof, even at the earliest stages proposed 
comparative anat^is? ^ * 

The first pnnople to keep in mind is that the more complex the 
entities (departments, institutions, state systerps) being compared, 
t^e greater the chance for error* There are more things to count and 
record when dealing with the more complex entities, and many 
more relationships* crossovers, and shared resources to accoun^for. 
For instance^ comparing institutions whose primary mission is 
instruction is easier than comparing inetitutioris whose mission in* 
volves major effects m research and public service as well as instruc* 
tion. Indeed,, we have noted that it may^e a practical impossibility 
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to iron out all of the discrepancies iWdata-recordingpracticfes among 
major research universities (Topping 1979), 

Second, issues posed for comparative analysis can range from the 
simple and straightforward to the complex and byzantine. Tuition 
rates usually ^anbe compared with little danger of misinterpretation 
or other sorts of error (altliough one must be wary of how '^general 
fees" are treated). But the chances are slim th^it one coutd compare 
expenditures for athletics at most institutions without encountering 
al! sorts of errors. Faculty workload is another inherently difftcult 
i&sue» it is surprisingly hard to measure in 3 manner that is error-free 
and that will not be haughtily impugned./* first consideration, 
therefore, is whether the issue in question can be dealt with in a 
straightforward manner on one*s own campus. If it cannot be, 
expect serious problems in attempting comparisoru. The bfoad 
maxijT\ is that almost every kind of ambiguity is pregnant^ith the 
possibility ofdata errors. The academic deans office seems to be part 
of the administration and the instpuctional area. Some students 

' insist ori majoring In more than one subject at a time. The computer 
is now used by everybody, but costs often are still aliocat^ as they 
were ki the days when only the science and math departments used 
it. Summer school may overlap two£scaI years. C<3un^eIors may cb 
both ^academic and personal couhsefing. Federal studenfaid fiands 
come into the institution and^re allotted to students, who there- 
upon give all or most of the money back to the institution. There is 
enough ambiguity Resent in each of the circumstanc& we have 
cited to lead people of soliod mind and good will to count and 
allocate in quite different ways- 
Third, mere are graduations in the *Vcondarine6s*4 of d^ta^. 
Strictly spealc^ngt use ofdata collected by another is secondary use, 
Ih^a prac'ticalVenset the prospective user may be fairly close to 
secondary data through jdirect contact with and knowledge of the 
person who collects the data,, or because the collectors have taken 

■ the time to adequately document what they did and the assump- 
tionS they ma^e, Orvthe other, hand, thejiser can be relatively far 
fi;omthe data even when they^ere generated at the user's own insti- 
tution. Consider a time-series comparison whe'h^^ORe remembers. 
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and there is no record to indicate ho\\' the early^ data were gathered, 
or how Larofiitly. Other things being equal* the cl^an(:e for Qtror is 
likely to increase with the "dista^e" t>^ween the user an^^ the data 
sourLe» partiuutarly* with respect to validity. So initial assessment of a 
proposed LOmparison involves the question; What do we know- 
about the proposed data squr^:e^ tn^:luding (:ollection procedures, 
mo'tjvation, and maintenance responsibilities? A corollary deter- 
minationnswK^^ther Ills possible and feasible to get closer to the data 
source, should that beSppropriate. * 

Finally, we recommend that*the entire data^luality issue be 
addressed within the framework of the intended use oft he compara- 
tive data- In this context, use can be taken in two s^nsesX)ne has to 
do with the /orm, or manner, in which the data are handled and 
presented- For instance, data accuracy will betess important if data 
will be 4sed in a highly aggregated form. Suppose one were to com- 
pare the (.han^e over time in the proportion <5f degrees awarded in 
the humanities by agroQp of^iberal-arts colleges. Even granted that 
common definitions are in use, some measurerrfent error is tikdy. 
But if the comparing college tumps the responses from the other 
institutions, and compares its own data with the aggregate data 
tmeans, medians, or wTiatever), jnaccuracies will tend to wash out^ 
barring some systematic error Similarly, comparative tiata from 
various departments or institutions can be used in a cross -sectional 
analysis, using techniques such as analysis of variance or linear 
* regression. In a regression approach, an institution can el-aluate its 
actual performance by comparing it to a performance levd predicted 
by the model (based on wnat the group asa \Aoledid). Again, inac- 
curacies m the data rend to average our. By contrast, head-to^head 
comparisons between departments or institutions or states usually 
will require a relatively higher degree of data quality* at least as 
regards accuracy and reliability. 

In the second sense in which the use and the quality of compara' 
tive data are related, quality perse is not affected by the use. Rather^ 
the use dictates^ or circumscribes, the level of quality needed. It is 
widely accepred thar Che characteris'tics of information (Useful dara) 
needed tosupport strategic planning are (lifferent from.those needed 

"—^ ■ ■ : 
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tu iuppurt management (:ontrol or operational (tfansa^:tional) 
aLtivitJCi (Gorry and S(:ott Morton 1971). At the strategic level, 
iiifurmation need not be highly acLurate or precise: knowing that 
enrollment is around 3,500 FTE students would be no less Refill 
^ than knc*itjng that the e,\aLt number was 3,524 on the census/date. 
At the operational le\cl, of Loursc, cheeiirollment of each and Vvery 
studcpt is duljM^corded, And at the management-control leveT, the ■ 
rc\ enuey^generated by the additional 24 students could have a 
mcanj^ifgful impact ^.?n Lash flow and budget balancing. Similarly, 
information for strategic planning usually can^tieJesa current and* 
less disaggregated than that employed for operations or manage- 
rnent Lontrol, By Lonsidering the level of decisionmaking that tfie 
analyi^is is intended to support in conjunction with 'the other 
elements of the assessment we have discussed^-it^ould be possible 
at the outset to estimate where the data^oroblems are likely, to be 
cnLOuntered, how severe they are likely to tJe, and whether they can 
be toig;ated. 



Quality of HEGISData 

Among the multitude of source^fcomparative data'j IS 
partiLularly well known and most wicTely used. The HEGJS system 
embraces eight national surveys, ^'hile data quality protaB)ly varies 
av^ros^ all of the different types of data-in the system, w^e will look at 
only three of the surveys— finances, enrollment, and earned 
degrees— because so far, they are the only ones chat have beeo 
evaluated at lengtn. 

The quality of HEGIS financial data has been studied from the 
perspCLtive of the states as well as from that of individual institu- 
tions. On the institutional side, one evaluative^stratigy has been to 
compare HEGIS data to audited data that were coded to AICPA- 
NACUBO standards, and then to ask two questions; Have the 
institutions provided valid data by following HEGIS insthictions 
and reporting wHat was requested? Are the HEGIS data accurate, 
reflecting correct counting and recording? The results are reported 
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\n Conger {1979), MmteTarilj Cogger (I979a,b,cK and Parrick and 
Collier tl979), and summarised in Stroup (1980). In^brtef^ there are 
two major findings. The first is that there is considerable discre- 
panL> between HEGIS financial data and data coded to AICPA- 
NACUBO standards. For example, for fiscal year 1977* abput 20 
percent of the 1 2 5 private institutions studied efther overreporced or 
underreported instructional expenditures by 15 percent or more. 
The figures were worse for academic support, student services, and 
institutional support but somewhat- better on the whole for data on 
revenues by source* such as tuition and gifts and grants (Klinter ani^ 
'Conger 1979a). The investigators conclude that disaggregated 
HEGIS financial data should be used with extreme caution. The 

y >^ond major finding was that HEGIS financial data in the aggre- 
gafe compare very favorably with the audited and coded data 

^^atrick and Collier 19?^. As one might expect, thereporting errors 
tend to cancel each other in th^bsence of systematic errorj^ 

Other studiesofthe quality ofHEGISfinanciat data from an insti- 
tutional perspective have relied on surveys of or intejviews with 
users of the data. The net result has beerv laund^ 1^ of typical 
ptobJems, such as the failare of institutions in systems to i^fclude 
jheir prorated share of the revenues and expenditures of the central 
administration, or the fact thsk NCESwill sometimes i^fer data for 
institutions that fail to respond to the suryey. (See Hy^ and Dick- 
meyer 1980 for more exam ple^ a s well as chapter 2 of t^js book.) On , 
the whole, this second gr<^^m studies leads to a sorKjwhat more 
sanguine viev\^ of the quality question— a general feeling that HEGIS 
data quality is improving {Andrew, Fortune, and McCluskey 1980). 

from the state perspective, most of the stody of financialdata has 
focu&e^on the ways in which^tate practices and structures are dif- 
ferent enough to undermine the comparability of HEGIS data from * 
one state to another. The most thorough account oftbedifferences^ 
and the ramifications for data qu^ity and comparability^^^n be 
found in Ryland ^1981). Overall, her study suggests tifet the differ-^ 
ences in state practices for fianding higher education (some of whieh x 
we noted in chapter 2) ar^ such that complete data comparability^ 
across states is n<ft achievablep^oreover, omissions ii> the HEGIS 
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data— cspe^:iallv regarding student aid, retitement benefits, and 
vocational edui: ation^result-in underestimates of state support for 
higbcT education. Gnce more, tt is well to proceed with care. 

The qualu^f HEGIS data looks somewhat better in the other 
two areas studied: enrollment anS degrees awarded. In the 
Vl'ESTAT study of these areas (Peng 1979), HEGI^data were com- 
pared to reconstructed data, interview responses, and audits of 
ort^lfial reports. The study indicates that the quality of chese two - 
- t> pes of HEGIS data IS generally quite good in the aggregate; error , 
rates were typically lest than 1 per^:enr Problems wereen^:ountered, 
howeven with disaggregated data of both types* On the enrollment 
side, more than 25 percent of the institutions studied did not follow 
KCES Lla&i^ificationsfor detailed student levels, distinguishing first- 
time fres5men was one (:ommon problem, especially for two-year 
^:olleges. (p^oOsisten^:y in the way institutions defined fiill- and part- 
time students was also notted/^'ith earHedflflegree daia, signifi^:ant 
problems were not found at the level of major fields (for example, 
biological S(:ien(:e)- But problems appeared when disaggregation was 
taken to the level of program categories (for example, molecular 
biology] within major fields. Double majors also were the source of 
in^:orisistencies in classification frbm one institution to another. 

As for the quality of HEGIS ^Jata in general, a survey of higher- 
edijtation institutions and state agencies showed that 85 per^:ent of 
the respondents felt that the accuracy of the data was acceptable or 
better (^ndrew et'aL 1980), In a related study by the same investi- 
gators, interviews with 75 higher -education researchers, financial 
' , officers, institutional researchers, and academic planners'W'ealed a ^ 
* solid consensus that HEGISdatavjere accurate enough at the aggre- 
gate levd.^'itivtfie exception of the financial survey, the data were 
dpe^ned accurate enough for analysis and comparisons down to the 
institutional level (with some reservations about aspects of data on 
ethnic membership,*part-time enrollments, and faculty salaries). 

On the vwflble, tlie quality of data from particular sources such as 
HEGIS and others -is a fetrly complicated mattg^F. Too much so, we 
" would argue, for mo^' admintstratgrs to tackle, considering their 
othpr responsibilities. But there'isa need for an institutional capacity 
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to address chts aspect data qualjiyj if comparacive data 3xq co be 
iiw?d\^ichLonfiden^:e.lDforniaaonspeLiahsi5— planners^ instiipcional 
rcii^ariwher^j and torm^cimes assistants to the dean^r president— are 
often the best persons towkeon the responsibihty for assessing data 
quafiiyi sin^wC it m^y wel! comj^fement cheir other dunes. 



State of the Analytic Art 



Although important te^hnic^ issues relafS to tht use of cornpara- 
tue data* we believe that from'an administrative perspective, the 
key issues art not so much^technical as managerial. Using compara- 
tue data,ort a ^:ampus ts a process that may misfire, even if the data 
are good and tKe technical analysis is sophisticated. Or it may 
\ aluabtv Support de(:isionmakingj even if the data are marginal and 
the anal>sfs barely adequate. To be su^:cessful, the ptocess needs to 
be managed and looked after. Turning it over to a technical staff^ 
liowe\er competent^ is unlikely to be sufficient. In this-^nd earlier 
(:hapters, \ve ha\e described at length the Comparative analysis and 
the administrator's role m it* We have only a few comments to add. 

Firstj stay al&reast of human interventions. That an investigation 
IS emptrKali based on data, certainly does not mean it ts tte>tally, 
objecti\^. It IS never enough merjely to let the fa^:ts speatrfor them- 
^Ives* Those who work with data kfto^w better^ especially in the 
cdntext of which we sj^eak- The reality of th&, highe>^du^:arion 
enterprise is &r richer, far more complicated^ than theine^ger data 
we have can.depict. Subjectivity, in the form of human interven- 
tion^ (:olors nearly alfdata analysis. "Wecouldn^t get rtiisy^ar*s data, 
so \ve used (ast years instead/* "We decided to make the cut here." 
"We chose toroe me4iansas the measure of central tendency." "We 
purked 1976 as jhebase year/*The administrator should be apprised 
of these inter^ning decbions en route. They will be tecKnicaf, in 
varying degt^e. But generally speaking, the choices ought to make 
s^se to the administrator, who should feel com^prtaole that the 
choices made do not jeopardize the project. 

Second, take all due care to optimi:e tjhe tmpact^of the analysis. 
Make sure that the effort is not wasted (while a(:knowledging that 
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tbtfre Will be those tvho would just as soon have it be other v^se< In 
thi^reg^ra; It iscssentiaho convev information in such a way that it 
^lU be understood correctly. Reporting thQ*findjngs of a compara- 
tive analv&is is not fundamentA^ifferent fri^m reporting empirical 
findirTgs generallv. But it r^jf ^(mre extra care to Snsure that the 
comparative data will not be misleading, Gomparative data are 
more likcly^to be misinterpreted because they usually are less 
familiar ti^fflj^ntial* users tfiaft data tba* refer to internal matters 
on[>< InterTpirdataare relatively well understood by rnost who work 
in the organisation^ obMOus errors* ttiU likely b^spotte^by veteran 
observers. who ha\e de\eIoped a stense of the normahrange for par- 
ticular variables. That intuitive sense is less reliable when applied to 
the ^^rformancc of ^ \ariety of other, upfamK^pr departments or 
mstitutio^s, ^ 

Spccificiillv, th? following sreps should be takejt; 

L Adhere to fca5ic data -pTi^smtanon standards. In^me ways, corp- 
parativc data arc especially liable to b^mg presenred in a mis- 
leading way. For example'i ah institution's performance will 
oftert be compared to a group normi which will be an average 
ofsome kind. If it happens to be a st3tisti<;^l/ff3iean, ir ?ssulyect 
to the effects of \ery large or \ery srriall values, especially jf the 
sample si;c is small— and* it often is, tn a comparative analysis. 
An obvious w^v around this problem is to present the 

* norm and rhc individual dita^^values. That ,wayi^readers^can 
judge for themselves how well the nbrm, represents the collet- 
tive beh3^aor in qu'cstion, , , ^ 

,2. Proitd*: approprmu background maienal. Three aspects of need 
cc^ju^re consideration. Firsti how familiar is the audtertce with 
higher-education tssuesi Higher-education ydatai siaiistKal 
routinesi and the like' Perhaps a general briefing js ne^kd< 

"^Second, whaf additional datairiight help thb^udience under- 
^anj^ a particular comparanve'relationshlb? For mstance^ 
suppo^ one wefe comparing changes inUverage faculty 

. salaries over a number of years at several insmutions. It wobl^ 

* help rhe audience underst^d the cpmipaT|ftive results if data 
<.uuld be provided on th|.pr^]t3ortions eifcontmuijjg and nevt 
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faculty each year at.each institution. Third, [he audience has a 
fight to know about the human interventions that affected the 
results, and about daCa and analytical problems that could be 
affecting che results. CavCciCSi (jualifications, and ocher explor- 
ations should be suitabl^y int^jgrate^wick the comparative 
* data, so trtat it \\ ill be hard to ignore or overlook rKem. It is noc 
unusual fo^ a report to have sec'^dary and certiary audiences. 
They too deser\e a more complete picture than the data alone 
is likely so provide, ! 

3. StUci an appropriate mode of pT€knui,Uon, Se\'eral cimes we have 
noted that daCa-do not become inftf^ation automatic ally. 
Indeed, information can get lost arritd too much data. So when 

' .pfanning presentation of a oprpparattve analysis, in which 
numbers are sure to abound, take time to consider what might 
be the most effective mode of presentation for the primary, in- 
tended audience, A board of trustees may have rather different , 
preferences in this regard from those of a legislative committee, 
or a faculty senaie. ^ j 

f J " ' ' 

To,con4.iude,\^e adv ocate a strong^managenal^ole for administra- 
tors in the^ development of a comparative analysis. We thinly they 
ought to be in\olved from beginning to enj. There are just too 
jTiany.potential problems to adopt a hai;jds^ff, la'lsse: faire attitude. ^ 
lathe end, the state of the artincomparahve analysis is'determined 
by adminisrra^fve^kill as.muchas.by technical virtuosity. . 
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Hereafter: 
Projections and 
Recommenda&)tis 



^ ^ rom ttie outset, \ve have tried co keep in mind Volcatre's obser- 
1l vation that *^he secrec of beinga bore isto tell everything." And 
the state of the art of prognqjtics being whac it is, we feel ^^oubly 
justified 11^ qukkentng the, pace of discpur&e at this point. In the 
remaining fe\v pages^ we indulge in hunches an<3 guesses and express, 
*■ soVne preferences for the fiature, abouc which our data are not 
; altogerfier complete. * ' . 



The 'National Outlook 



The federal dara-colleccion effort On higher education b^'an 
around 1870, then took quantum leap forward in*1929-30 under 
the auspices'of the U.S. Office of Education.The next major advance 
came in che mid-ft60s with the HEGIS effort described,.in the 
preceding chapter. AobtKec^jgnificant contribution to the nacfionat 
data pool, che National Science Foundatioiv Surveys of higher- 
education institu|(OTiSt began in the 1950s, Togedier^ che tlE(5lS^ 
and NSF files consrnute a substantial resource for comparative data* 
Whac does the fucure hold for tho^e systems?. 
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There is little basis <i>n which to construct long^nge forecasts. In 
the near term, however, it appears that the NS? data-collection 
effort uill ^:antinije in i(s current form^ while HEGIS wttl be under 
Some pressure^ to ^:on(ract in scope. The pressure to restrKt HEGlS 
rcflcLts the desire to hold down federal expenditures and reduce the 
amount ofp^per-work imposed by the federblgo\ernm^t. Thus far, 
theeffc^:tsofnthe HEGIS system have beeij modest, A major survey, 
on instTtunonal ^: ha ra^:t eristics, was eliminated in 1982-83, buc 
aoothcr, on state^eveles^ienditures, was added. One or more addi; 
tronal surveys may be dropped, but ^ve think it more likely chat they 
will be conducted less frequently. There 'are advocaresboth for and 
against che present configt^ration of the HEGIS system. Support is 
\iidesprcad fcffc- rfce baSK proposition tha^ rhe federal government 
should play a role in gathering statistics on higher «^ucattdn. 
The debate mainly 1:001: erns how much and for wjiat purposes. In 
any^vent, it seems most unlikely that the HEC3S system v^ll be 
sjgnificantiy expanded, , ^ * ' ^ 

Other forms of higher-education data coHecrion on a national 
scope are likely to continue to play a supportive yet impoYta'nt role, 
Vl'ide interest in enrollm^,, faculty salaries, tuirion, and^finances 
should ensure that dat^witl be coUec*fed annually from at least^a 
sample, if nor ^il, of the nation s colleges and universities. Some 
chan'ges <:An be^xpected* Ofi occasion it ma^ be just a case of pass- 
ing the responsibilij^y for rhe survey Irom one^Organifiation to 
another, ^or example, the enrollment survey conducred for many 
years by me University of Cincirtn^ti will be done in the future by 
the Uriive^iry qf AIabar^^a*And neU' surveys appear from time'to 
time. An example is the salary survey by the College and University 
Personnel A^^iation (CUPA) and rhe. American Association of 
State Coydges and Universities (^SCUy initiated in^ 1982^ The 
ouUook 15 not ennrely positivej^iowever. Some renr surveys may 
be curtail^ be^:ause of firiancia! constraiQti. Nationa^survtys are 
sometimes peripheral to ttie mission of the spdn^dng agency. For 
ThstanLe, rhe Washington Coordinating Boar^^ for Higher Edjuca- 
tion conducrs an^arinual tuttibn sur^^ey as a means of j;enerating 
ba^tground data tO^Aform tuit ion -ser ring in that state. Continued 
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PROJECTIONS AND, RECOMMENt>AJlC>NS 



severe St a re -level finan6al constraints could put ^uch Survey effort, 
^tn jeopardy.^- ^ ^ . . ' 



utl^fe 



Tfie State^l-evel O 



Far-reaching and uadespread fiigher -education data coltec|ion at 
h the s^te level is a post-World ^'ar*Uphenomer\oTi. It wasfueledVor 
^ the ijfiost part by tremendous growth in the enterprise* For most 
statesV but not all, this growth is over, at le^Snb^ this century* 
Enpllm^ht d^lines already 'begun in some-states are expecteS tb 
' contmue apace through this decade* Ironically, severe tIcc line 
creates as mQch need for comparative data as growth creates^and , 
per haps"* more* - * * 

The need for (:omparative*'3ata will be feh strongly m states that 
encounter ^:ontinuing finan(:laldifficulties, regardless of tneir enroll* 
^ ment e?tperience. Sustained national econo^nic reco\?ery presum- 
' ably WtKi:elicve ^he strain in most states* Shon of that, many states 
will face h3?d choices between* higher education and otiier sbcteil 
services, ara^ithin the^anq^Iy ef servicers offered by their systems 
"of highgggducation.^ot^parative data will besought as a means of 
rauop^^ng th^se. choices or justifying those made on ^other 
grounds. Probably the only impetu&.acting.against data collection 
ur£ler these circumstances will be the cost of the collecaon and 
analysis effort^ ' : * 



The Institutional Outlook 



With reject to fndividual colleges and univefsities, theprospects 
for the fumre use of com^ai^tiiJ^ data are intriguing* The demand 
for such data Ukely will remain high, for both management dontroL 
and strategic planning. We cfo not foreseg^sdbstantiat change in the 
yn3ertvingcjyhamics operating on the current need for comparative 
data- . ^ . * ^ . . 

Granted th'at continuing need, 'developments will focus on i^e 
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form of usg and th^ mode of transfer of comparative data. Most ■ 
administrators today are likely to find comparative (^ta, along with 
other daaa they themselvis use, residing on a sheet of paper. We 
expecvhat in th^tur^j the data will reaide instead in a micropro- 
cessor. For som«, mis will amount to little more than a change in the 
record -keeping system. Others, though, will ^recognize the wider 
opportunities afforded by these^new electronic tools— in particular, 
the ability to manipulate data and relati^onships among data in a 
dynari^ic mode. "What if games, evaluation of multiple strategies,* 
and so forth will become as available to the typical administrator as 
'the top^^rlg machine is no^V. 

Will comparative data turn up in microprocessors dedicated to 
high-level administrative u&es? One would hope so. What we envi- 
sioj) Js admmistKators routinely looking beyond their institution's ; 

^ operating data ba^,to get information. Once o'btafned, external 
data can be mer^d 'wiCh aggregated internal data to support 
analyses helpfial to decisioiimaking. We do*" not wish to oversell the 
ne\v electronic tools orjfor that matter^ the data that may be 

* manipulated with them. There simply is no substitute foi; good judg-^ 
mem on the part of those principally responsible for the well-being 
of our colleges and universities. But the decisionmaking process * 
shoi^ld be supported as^best we are able. The new tools can help 
itvthis regard. And in the process^ they will enhatice the utility of 
comparative data. 

The iriicroprocessor will prove to be mor^e than a new Jiost for 
data. It offers the prospect foi*great improvements in data transfer, 
Electromc linkages aniong computers {or terminals) creates a new 
situation with reepecf to both networki,ng and access to central data 
^es. Networks among institutions for the*purpose ofsTiaring^^ 
can become more inter^crive^ faster^ and more responsive: to the' 
-exigent needs of the decisiotmaking process than the best arrange- 
ments of the past, Thie ^ame can be said for central data bases: even 
as we wrttei the list of oft -line data Bases grow5 daily. Much the 
data currenty avaifabkis^more suitable to research than to adminis- 
trative purposes, but it is just a matter of time before that situation is 
corrected. Actually, some opportunities afre^y available, such as 



86' 




PRQ]ECTI0N8 AND R£COM\fENDATION^ 



electronic access totheNea YorkTimeSy have utility forth^drrJinis 
trator in higher education. In store is electronic access to data bases^ 
dedicated specifically to higher education^^ik^ th'^e mairtteinedby 
NCES^nd NCHEMS. Indeed, EDUCOM Has already begun an 
ekcfronic data-sharing ser\ice on a ttial basis* Patticipating institu- 
tions share financial data with one another electronically^tn efifect 
creating their own comparative data base^ while simultaneously 
gaming access, to a computet -based rnod^ling tool (EFPM) with 
which to analyze and manipfulate the data* 



What to Do 



./ 



V 



A basic premise of thfe book' has been tha^^mparative data are 
here to stay- They will be used* Administrators will continue ro be 
asked to supply data for comparative purposes; rhey^ill find them- 
seKes having to explain and irtterpr^ such data^with respect ro t(\eir 
own c|jpartments or institutions* On these assumpftons^ it seems ' 
sensible for administrators to take at leait some modest st^ps ro 
enharKe th^ utility and minimize the risks of using comparative dara* 

The task can be addressed from two perspectives— that of the user , 
of comparative data on one's own campus, and t hatof a participant ^ 
in the larger state and national compararive-data efforts. From rhe 
iristitutional point of view, several issues arie worrh considering. First 
there is ^he matter of a general srance toward comparative data. 
Despite all we have-said about its virrues, we would not claim rhat 
compararive data are rypically "of the essence** or that they should, 
be ubiqtetous- Only tarely, as ih determining comparative advan- 
tages, are such data criiciar More often^ they are a sdurce of back- 
'ground information rhar can improve inrerpretation. For manage- 
ment purposes, it does nor make much sense ro round up a lor of 
compararive data for their own sake. If -we have learned anything 
over rhe years, iris thar lots of data do not n^ecessarily translate rnro 
lots of information- The sound strategyfis ro choose wit^i care rhe 
issues on\whic^ comparative data are ro be garhe'red arui used. 
Discourage shotgun approaches* 
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A second internal strategy concerns building and maintainiifb^ 
capacity to acquirt^ comparative data ^nd co do comparative 

^nalysi^. As a foundation, resources have to be assembled a^ftii^iain' 
tained* see great utility in considering the de^^elopmert^ of this 
particular capacity as p^t of cne more general process of providing 
information for management. We follow Jon^s (1982) and others in 
thinking that an institution can best come to grips with that process 
bv ^ocysing bn the role ^f informatior> specialists. However^ the t«e 
ofcomparativedataonthe campfis should be viewed asesyentikll^^ 
management task,"under administrative control. J ^ 
From an external per^frctive^ it nijust seem that the inrt4tutional^ 
administrator can dy little but wage a rear-guard artion^ trying'to 
keep the data requirements of external agencies, and their attendant 
costs, to a minimum (Floyd 1982). But perhaps the p^riodof'consoli- 
dation wje ate entering offers an opportunity fDr institutional officiak 

*to exerr^other K^nd of influence. Essentially^ they should *dvycate' 
a f«iv basic principles. First, the inclusion of any data element m the 

, txiprnal s^ystem should be justifiable on managemenr grounds. That 
IS, itshQjuld be reasonably ol^ous what sort of Trrformation istobe^ 
gleaned from the data provided. And the. adequacy of the data 
system ^to^support r^anagpi«P*&ife (including policy analysis) is'the 
propej issue, rathet^*S(an its copiprehensive'neiS per se. Second^ 
maintaining da*s!^5uatity should be a shared rfesj^pnsibility. Institu- 
tiOT>al officials c^i^nsist t hat ^ External agencies make adequate 
efforts to ensure tfet the da(a they collect meet appropriate 
standards for yalidityt^eliabllity^ and accuracy. At the same time, 

^the institutions therriselves shoi4ld be willing to assist in lilentifying 
data quality problems (Hyatt 1982), They have mucb-to gain by 
carefully attending to their own provis^ of data to those ejtternal^^ 
systems. Too often^Jthe source of bad data about an institution is the 
institution itsel^Third, institutions should insist oTi a usefully <ym'" 
plete ana prompt return flow oif data from th^-cQllection agency. 
Closing tW loop will give all concerned a larger stake in the data, 
thus enhancing prospects fDr alality and for maximizmg the utility 
of tl\e compat9tive<lata resoir^e, | 



PROJECTIONS AND RECOMMENDATIONS 

To' Shakespeare, cOmQarisons werf "odorous^" and to John ^ ^ 
L>dgate (\\rittng in another penpid of want fo^^^highet education) 
they \\erc a source of ''gret*grevaunce." We prefer to end this $mair 
boojt on the note funded by Ivan Petrovich Pavlovi ] 

' "Learn, compare, collect the facts!"* I 
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